You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Giordon Stark (Jira)" <ji...@apache.org> on 2021/06/15 15:56:00 UTC
[jira] [Created] (ARROW-13085) Apache Arrow minimal cpp build
segfaults with pyarrow libs
Giordon Stark created ARROW-13085:
-------------------------------------
Summary: Apache Arrow minimal cpp build segfaults with pyarrow libs
Key: ARROW-13085
URL: https://issues.apache.org/jira/browse/ARROW-13085
Project: Apache Arrow
Issue Type: Bug
Components: C++, Python
Affects Versions: 4.0.1
Environment: See dockerfile: https://github.com/kratsg/awkward-arrow-cmake-pybind11/blob/0e196983b2452c4c3dbbde40984db7031397d0a7/Dockerfile
Reporter: Giordon Stark
h6. _Refer to [https://github.com/kratsg/awkward-arrow-cmake-pybind11/issues/6#issuecomment-861592692] and code-base available here: [https://github.com/kratsg/awkward-arrow-cmake-pybind11/tree/0e196983b2452c4c3dbbde40984db7031397d0a7]_
The minimal build example provided by arrow ([https://github.com/apache/arrow/blob/master/cpp/examples/minimal_build/example.cc] [)|https://github.com/apache/arrow/blob/master/cpp/examples/minimal_build/example.cc] works fine using the libs shipped with {{yum}} on CentOS7. Using the libs shipped by {{pip install pyarrow}} instead provides a segfault at runtime. Here is the gdb dump with debugging flags included
{code:java}
[root@dab7bb04d93f examples]# gdb ./../build/example_arrow
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /Users/kratsg/awkward-arrow-cmake-pybind11/build/example_arrow...done.
(gdb) run
Starting program: /Users/kratsg/awkward-arrow-cmake-pybind11/examples/./../build/example_arrow
warning: Error disabling address space randomization: Operation not permitted
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7f9c58dff700 (LWP 1643)]
* Reading CSV file 'test.csv' into table
Program received signal SIGSEGV, Segmentation fault.
__destroy<std::basic_string<char>*> (__last=<optimized out>, __first=0x3200000000) at /usr/include/c++/4.8.2/bits/stl_construct.h:103
103 std::_Destroy(std::__addressof(*__first));
Missing separate debuginfos, use: debuginfo-install glibc-2.17-324.el7_9.x86_64 libgcc-4.8.5-44.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64
(gdb) bt
#0 __destroy<std::basic_string<char>*> (__last=<optimized out>, __first=0x3200000000) at /usr/include/c++/4.8.2/bits/stl_construct.h:103
#1 _Destroy<std::basic_string<char>*> (__last=<optimized out>, __first=<optimized out>) at /usr/include/c++/4.8.2/bits/stl_construct.h:126
#2 _Destroy<std::basic_string<char>*, std::basic_string<char> > (__last=0x0, __first=<optimized out>) at /usr/include/c++/4.8.2/bits/stl_construct.h:151
#3 ~vector (this=0x7ffff43602a8, __in_chrg=<optimized out>) at /usr/include/c++/4.8.2/bits/stl_vector.h:415
#4 arrow::csv::ConvertOptions::~ConvertOptions (this=0x7ffff4360220, __in_chrg=<optimized out>)
at /usr/local/venv/lib/python3.8/site-packages/pyarrow/include/arrow/csv/options.h:64
#5 0x000000000040171a in (anonymous namespace)::RunMain (argv=<optimized out>, argc=<optimized out>, this=<optimized out>, this=<optimized out>)
at /Users/kratsg/awkward-arrow-cmake-pybind11/src/example/arrow.cpp:39
#6 0x000000000040135f in main (argc=<optimized out>, argv=<optimized out>) at /Users/kratsg/awkward-arrow-cmake-pybind11/src/example/arrow.cpp:66 {code}
Below is the full CMake log + build log
{code:java}
# ./build.sh
-- The CXX compiler identification is GNU 4.8.5
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- pybind11 v2.6.2
-- Found PythonInterp: /usr/local/bin/python3.8 (found version "3.8.10")
-- Found PythonLibs: /usr/local/lib/libpython3.8.so
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- PyArrow includes : /usr/local/venv/lib/python3.8/site-packages/pyarrow/include
-- PyArrow libraries : arrow;arrow_python
-- PyArrow library dirs : /usr/local/venv/lib/python3.8/site-packages/pyarrow
-- Awkward includes : /usr/local/venv/lib/python3.8/site-packages/awkward/include
-- Awkward libraries: /usr/local/venv/lib/python3.8/site-packages/awkward
-- Libraries: /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward-cpu-kernels.so /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward.so /usr/lib64/libdl.so /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow.so /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow_python.so
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/kratsg/awkward-arrow-cmake-pybind11/build
-- pybind11 v2.6.2
-- PyArrow includes : /usr/local/venv/lib/python3.8/site-packages/pyarrow/include
-- PyArrow libraries : arrow;arrow_python
-- PyArrow library dirs : /usr/local/venv/lib/python3.8/site-packages/pyarrow
-- Awkward includes : /usr/local/venv/lib/python3.8/site-packages/awkward/include
-- Awkward libraries: /usr/local/venv/lib/python3.8/site-packages/awkward
-- Libraries: /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward-cpu-kernels.so /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward.so /usr/lib64/libdl.so /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow.so /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow_python.so
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/kratsg/awkward-arrow-cmake-pybind11/build
-- Cache values
CMAKE_BUILD_TYPE:STRING=RelWithDebInfo
CMAKE_INSTALL_PREFIX:PATH=/usr/local
CPU-KERNELS:FILEPATH=/usr/local/venv/lib/python3.8/site-packages/awkward/libawkward-cpu-kernels.so
LIBARROW:FILEPATH=/usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow.so
LIBARROW_PYTHON:FILEPATH=/usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow_python.so
LIBAWKWARD:FILEPATH=/usr/local/venv/lib/python3.8/site-packages/awkward/libawkward.so
LIBDL:FILEPATH=/usr/lib64/libdl.so
PYBIND11_FINDPYTHON:BOOL=OFF
PYBIND11_INSTALL:BOOL=OFF
PYBIND11_NOPYTHON:BOOL=OFF
PYBIND11_PYTHON_VERSION:STRING=
PYBIND11_TEST:BOOL=OFF
###################################################################
######################### BUILD LOG BELOW #########################
###################################################################
/usr/local/bin/cmake -S/Users/kratsg/awkward-arrow-cmake-pybind11/src -B/Users/kratsg/awkward-arrow-cmake-pybind11/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/bin/cmake -E cmake_progress_start /Users/kratsg/awkward-arrow-cmake-pybind11/build/CMakeFiles /Users/kratsg/awkward-arrow-cmake-pybind11/build//CMakeFiles/progress.marks
/usr/bin/gmake -f CMakeFiles/Makefile2 all
gmake[1]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
/usr/bin/gmake -f CMakeFiles/babel.dir/build.make CMakeFiles/babel.dir/depend
/usr/bin/gmake -f CMakeFiles/example_arrow.dir/build.make CMakeFiles/example_arrow.dir/depend
/usr/bin/gmake -f CMakeFiles/example.dir/build.make CMakeFiles/example.dir/depend
gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
cd /Users/kratsg/awkward-arrow-cmake-pybind11/build && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build/CMakeFiles/babel.dir/DependInfo.cmake --color=
gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
cd /Users/kratsg/awkward-arrow-cmake-pybind11/build && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build/CMakeFiles/example_arrow.dir/DependInfo.cmake --color=
gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
cd /Users/kratsg/awkward-arrow-cmake-pybind11/build && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build/CMakeFiles/example.dir/DependInfo.cmake --color=
gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
/usr/bin/gmake -f CMakeFiles/example.dir/build.make CMakeFiles/example.dir/build
gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
/usr/bin/gmake -f CMakeFiles/babel.dir/build.make CMakeFiles/babel.dir/build
gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
/usr/bin/gmake -f CMakeFiles/example_arrow.dir/build.make CMakeFiles/example_arrow.dir/build
gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
[ 14%] Building CXX object CMakeFiles/example.dir/example/python.cpp.o
/usr/bin/c++ -Dexample_EXPORTS -isystem /Users/kratsg/awkward-arrow-cmake-pybind11/src/pybind11/include -isystem /usr/local/venv/lib/python3.8/site-packages/awkward/include -O2 -g -DNDEBUG -fPIC -fvisibility=hidden -std=gnu++11 -MD -MT CMakeFiles/example.dir/example/python.cpp.o -MF CMakeFiles/example.dir/example/python.cpp.o.d -o CMakeFiles/example.dir/example/python.cpp.o -c /Users/kratsg/awkward-arrow-cmake-pybind11/src/example/python.cpp
[ 28%] Building CXX object CMakeFiles/babel.dir/babel/python.cpp.o
/usr/bin/c++ -Dbabel_EXPORTS -isystem /Users/kratsg/awkward-arrow-cmake-pybind11/src/pybind11/include -isystem /usr/local/venv/lib/python3.8/site-packages/pyarrow/include -isystem /usr/local/venv/lib/python3.8/site-packages/awkward/include -O2 -g -DNDEBUG -fPIC -fvisibility=hidden -std=gnu++11 -MD -MT CMakeFiles/babel.dir/babel/python.cpp.o -MF CMakeFiles/babel.dir/babel/python.cpp.o.d -o CMakeFiles/babel.dir/babel/python.cpp.o -c /Users/kratsg/awkward-arrow-cmake-pybind11/src/babel/python.cpp
[ 42%] Building CXX object CMakeFiles/example_arrow.dir/example/arrow.cpp.o
/usr/bin/c++ -isystem /usr/local/venv/lib/python3.8/site-packages/pyarrow/include -O2 -g -DNDEBUG -std=gnu++11 -MD -MT CMakeFiles/example_arrow.dir/example/arrow.cpp.o -MF CMakeFiles/example_arrow.dir/example/arrow.cpp.o.d -o CMakeFiles/example_arrow.dir/example/arrow.cpp.o -c /Users/kratsg/awkward-arrow-cmake-pybind11/src/example/arrow.cpp
[ 57%] Linking CXX executable example_arrow
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/example_arrow.dir/link.txt --verbose=1
/usr/bin/c++ -O2 -g -DNDEBUG CMakeFiles/example_arrow.dir/example/arrow.cpp.o -o example_arrow -Wl,-rpath,/usr/local/venv/lib/python3.8/site-packages/pyarrow /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow.so
gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
[ 57%] Built target example_arrow
[ 71%] Building CXX object CMakeFiles/example.dir/example/minimal.cpp.o
/usr/bin/c++ -Dexample_EXPORTS -isystem /Users/kratsg/awkward-arrow-cmake-pybind11/src/pybind11/include -isystem /usr/local/venv/lib/python3.8/site-packages/awkward/include -O2 -g -DNDEBUG -fPIC -fvisibility=hidden -std=gnu++11 -MD -MT CMakeFiles/example.dir/example/minimal.cpp.o -MF CMakeFiles/example.dir/example/minimal.cpp.o.d -o CMakeFiles/example.dir/example/minimal.cpp.o -c /Users/kratsg/awkward-arrow-cmake-pybind11/src/example/minimal.cpp
[ 85%] Linking CXX shared module example.cpython-38-x86_64-linux-gnu.so
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/example.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O2 -g -DNDEBUG -shared -o example.cpython-38-x86_64-linux-gnu.so CMakeFiles/example.dir/example/python.cpp.o CMakeFiles/example.dir/example/minimal.cpp.o -Wl,-rpath,/usr/local/venv/lib/python3.8/site-packages/awkward /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward-cpu-kernels.so /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward.so /usr/lib64/libdl.so
[100%] Linking CXX shared module babel.cpython-38-x86_64-linux-gnu.so
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/babel.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O2 -g -DNDEBUG -shared -o babel.cpython-38-x86_64-linux-gnu.so CMakeFiles/babel.dir/babel/python.cpp.o -Wl,-rpath,/usr/local/venv/lib/python3.8/site-packages/pyarrow:/usr/local/venv/lib/python3.8/site-packages/awkward /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow.so /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow_python.so /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward-cpu-kernels.so /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward.so /usr/lib64/libdl.so
gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
[100%] Built target example
gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
[100%] Built target babel
gmake[1]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
/usr/local/bin/cmake -E cmake_progress_start /Users/kratsg/awkward-arrow-cmake-pybind11/build/CMakeFiles 0 {code}
One can reproduce this by cloning the corresponding repository recursively, then building the docker image, compiling the code, and then running it
{code:java}
git clone --recursive git@github.com:kratsg/awkward-arrow-cmake-pybind11.git
make build-image-debug
make debug # launches docker image
> ./build.sh
> ./run_example.sh{code}
where {{>}} denotes commands run inside the docker image.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)