You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Rares Vernica <rv...@gmail.com> on 2022/05/22 04:55:15 UTC
[C++] parquet::arrow::WriteTable process exited with code 0177
Hello,
We have a plugin that writes both Arrow and Parquet format files. We are
experiencing an issue when using Parquet format, while Arrow format works
just fine. More exactly, the process crashes in parquet::arrow::WriteTable.
Using gdb we identified the line when the process crashes
https://github.com/Paradigm4/bridge/blob/8c34f3b940b10a38eb8900cd05d7ee6aa7c082b0/src/PhysicalXSave.cpp#L494
The gdb output is:
(gdb) catch syscall exit_group
Catchpoint 1 (syscall 'exit_group' [231])
(gdb) c
Continuing.
[Switching to Thread 0x7f104b7f6700 (LWP 2363)]
Catchpoint 1 (call to syscall exit_group), 0x00007f105c031901 in _exit ()
from /lib64/ld-linux-x86-64.so.2
(gdb) bt
#0 0x00007f105c031901 in _exit () from /lib64/ld-linux-x86-64.so.2
#1 0x00007f105c0276ba in _dl_signal_error () from
/lib64/ld-linux-x86-64.so.2
#2 0x00007f105c02776e in _dl_signal_cerror () from
/lib64/ld-linux-x86-64.so.2
#3 0x00007f105c0229aa in _dl_lookup_symbol_x () from
/lib64/ld-linux-x86-64.so.2
#4 0x00007f105c026dee in _dl_fixup () from /lib64/ld-linux-x86-64.so.2
#5 0x00007f105c02eaaa in _dl_runtime_resolve_xsavec () from
/lib64/ld-linux-x86-64.so.2
#6 0x00007f1051926a88 in scidb::ArrowWriter::finalize
(this=this@entry=0x7f104b7f2e40,
arrowBuffer=std::shared_ptr (empty) 0x0) at PhysicalXSave.cpp:494
#7 0x00007f105192802f in scidb::ArrowWriter::writeArrowBuffer
(this=this@entry=0x7f104b7f2e40, chunkIters=std::vector of length 1,
capacity 1 = {...}, coordMapper=...,
arrowBuffer=std::shared_ptr (empty) 0x0) at PhysicalXSave.cpp:280
#8 0x00007f105192b35e in scidb::PhysicalXSave::execute
(this=0x7f1034004f00, inputArrays=..., query=warning: RTTI symbol not found
for class 'std::_Sp_counted_ptr_inplace<scidb::Query,
std::allocator<scidb::Query>, (__gnu_cxx::_Lock_policy)2>'
warning: RTTI symbol not found for class
'std::_Sp_counted_ptr_inplace<scidb::Query, std::allocator<scidb::Query>,
(__gnu_cxx::_Lock_policy)2>'
std::shared_ptr (count 8, weak 8) 0x8f348e0) at PhysicalXSave.cpp:879
#9 0x0000000000af71b7 in
scidb::PhysicalOperator::executeWrapper(std::vector<std::shared_ptr<scidb::Array>,
std::allocator<std::shared_ptr<scidb::Array> > >&,
std::shared_ptr<scidb::Query>) ()
#10 0x0000000000b3bc3d in
scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::PhysicalQueryPlanNode>
const&, std::shared_ptr<scidb::Query> const&, int) ()
#11 0x0000000000b3c1b5 in
scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::Query> const&) ()
#12 0x0000000000a0dd49 in
scidb::ServerMessageHandleJob::handleExecutePhysicalPlan() ()
#13 0x0000000000a0ee23 in
scidb::ServerMessageHandleJob::handlePreparePhysicalPlan() ()
#14 0x0000000000a0addf in scidb::ServerMessageHandleJob::run() ()
#15 0x0000000000a32ed8 in
scidb::Job::executeOnQueue(std::weak_ptr<scidb::WorkQueue>&,
std::shared_ptr<scidb::SerializationCtx>&) ()
#16 0x0000000000a6e938 in scidb::WorkQueue::WorkQueueJob::run() ()
#17 0x0000000000a32ff0 in scidb::Job::execute() ()
#18 0x0000000000a3fefc in scidb::Thread::_threadFunction() ()
#19 0x0000000000a409b9 in scidb::Thread::threadFunction(void*) ()
#20 0x00007f1058fc8ea5 in start_thread () from /lib64/libpthread.so.0
#21 0x00007f1057ea7b0d in clone () from /lib64/libc.so.6
(gdb) c
Continuing.
[Thread 0x7f10525ff700 (LWP 2355) exited]
[Thread 0x7f104bbf8700 (LWP 2361) exited]
[Thread 0x7f104b9f7700 (LWP 2362) exited]
[Thread 0x7f104b7f6700 (LWP 2363) exited]
[Thread 0x7f104b5f5700 (LWP 2364) exited]
[Thread 0x7f104b3f4700 (LWP 2365) exited]
[Thread 0x7f104abf3700 (LWP 2371) exited]
[Thread 0x7f104a9f2700 (LWP 2372) exited]
[Thread 0x7f104a7f1700 (LWP 2373) exited]
[Thread 0x7f104a5f0700 (LWP 2374) exited]
[Inferior 1 (process 2351) exited with code 0177]
Any ideas on what could be the culprit?
Thank you!
Rares
Re: [C++] parquet::arrow::WriteTable process exited with code 0177
Posted by Rares Vernica <rv...@gmail.com>.
Thank you, the hint about not seeing Parquet frames helped me solve the
issue. I was not linking the Parquet .so with the plugin .so. Once linked
everything worked as expected.
Thank you,
Rares
On Sat, May 21, 2022 at 10:45 PM Micah Kornfield <em...@gmail.com>
wrote:
> Hi Rares,
> Based on the stack trace, I'm not seeing any Arrow/Parquet related frames.
>
> Two things come to mind:
> 1. This seems like it might be heap corruption (see
>
> https://stackoverflow.com/questions/10578315/segmentation-fault-in-dl-runtime-resolve
> ).
> Try running using ASAN or valgrind.
> 2. (Less likely) It could have something to do with whether exceptions
> have been compiling the parquet library (Arrow catches exceptions and
> translates them to statuses).
>
>
> -Micah
>
> On Sat, May 21, 2022 at 9:55 PM Rares Vernica <rv...@gmail.com> wrote:
>
> > Hello,
> >
> > We have a plugin that writes both Arrow and Parquet format files. We are
> > experiencing an issue when using Parquet format, while Arrow format works
> > just fine. More exactly, the process crashes in
> parquet::arrow::WriteTable.
> >
> > Using gdb we identified the line when the process crashes
> >
> >
> https://github.com/Paradigm4/bridge/blob/8c34f3b940b10a38eb8900cd05d7ee6aa7c082b0/src/PhysicalXSave.cpp#L494
> >
> > The gdb output is:
> >
> > (gdb) catch syscall exit_group
> > Catchpoint 1 (syscall 'exit_group' [231])
> > (gdb) c
> > Continuing.
> > [Switching to Thread 0x7f104b7f6700 (LWP 2363)]
> >
> > Catchpoint 1 (call to syscall exit_group), 0x00007f105c031901 in _exit ()
> > from /lib64/ld-linux-x86-64.so.2
> > (gdb) bt
> > #0 0x00007f105c031901 in _exit () from /lib64/ld-linux-x86-64.so.2
> > #1 0x00007f105c0276ba in _dl_signal_error () from
> > /lib64/ld-linux-x86-64.so.2
> > #2 0x00007f105c02776e in _dl_signal_cerror () from
> > /lib64/ld-linux-x86-64.so.2
> > #3 0x00007f105c0229aa in _dl_lookup_symbol_x () from
> > /lib64/ld-linux-x86-64.so.2
> > #4 0x00007f105c026dee in _dl_fixup () from /lib64/ld-linux-x86-64.so.2
> > #5 0x00007f105c02eaaa in _dl_runtime_resolve_xsavec () from
> > /lib64/ld-linux-x86-64.so.2
> > #6 0x00007f1051926a88 in scidb::ArrowWriter::finalize
> > (this=this@entry=0x7f104b7f2e40,
> > arrowBuffer=std::shared_ptr (empty) 0x0) at PhysicalXSave.cpp:494
> > #7 0x00007f105192802f in scidb::ArrowWriter::writeArrowBuffer
> > (this=this@entry=0x7f104b7f2e40, chunkIters=std::vector of length 1,
> > capacity 1 = {...}, coordMapper=...,
> > arrowBuffer=std::shared_ptr (empty) 0x0) at PhysicalXSave.cpp:280
> > #8 0x00007f105192b35e in scidb::PhysicalXSave::execute
> > (this=0x7f1034004f00, inputArrays=..., query=warning: RTTI symbol not
> found
> > for class 'std::_Sp_counted_ptr_inplace<scidb::Query,
> > std::allocator<scidb::Query>, (__gnu_cxx::_Lock_policy)2>'
> > warning: RTTI symbol not found for class
> > 'std::_Sp_counted_ptr_inplace<scidb::Query, std::allocator<scidb::Query>,
> > (__gnu_cxx::_Lock_policy)2>'
> > std::shared_ptr (count 8, weak 8) 0x8f348e0) at PhysicalXSave.cpp:879
> > #9 0x0000000000af71b7 in
> >
> >
> scidb::PhysicalOperator::executeWrapper(std::vector<std::shared_ptr<scidb::Array>,
> > std::allocator<std::shared_ptr<scidb::Array> > >&,
> > std::shared_ptr<scidb::Query>) ()
> > #10 0x0000000000b3bc3d in
> >
> >
> scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::PhysicalQueryPlanNode>
> > const&, std::shared_ptr<scidb::Query> const&, int) ()
> > #11 0x0000000000b3c1b5 in
> > scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::Query> const&)
> ()
> > #12 0x0000000000a0dd49 in
> > scidb::ServerMessageHandleJob::handleExecutePhysicalPlan() ()
> > #13 0x0000000000a0ee23 in
> > scidb::ServerMessageHandleJob::handlePreparePhysicalPlan() ()
> > #14 0x0000000000a0addf in scidb::ServerMessageHandleJob::run() ()
> > #15 0x0000000000a32ed8 in
> > scidb::Job::executeOnQueue(std::weak_ptr<scidb::WorkQueue>&,
> > std::shared_ptr<scidb::SerializationCtx>&) ()
> > #16 0x0000000000a6e938 in scidb::WorkQueue::WorkQueueJob::run() ()
> > #17 0x0000000000a32ff0 in scidb::Job::execute() ()
> > #18 0x0000000000a3fefc in scidb::Thread::_threadFunction() ()
> > #19 0x0000000000a409b9 in scidb::Thread::threadFunction(void*) ()
> > #20 0x00007f1058fc8ea5 in start_thread () from /lib64/libpthread.so.0
> > #21 0x00007f1057ea7b0d in clone () from /lib64/libc.so.6
> > (gdb) c
> > Continuing.
> > [Thread 0x7f10525ff700 (LWP 2355) exited]
> > [Thread 0x7f104bbf8700 (LWP 2361) exited]
> > [Thread 0x7f104b9f7700 (LWP 2362) exited]
> > [Thread 0x7f104b7f6700 (LWP 2363) exited]
> > [Thread 0x7f104b5f5700 (LWP 2364) exited]
> > [Thread 0x7f104b3f4700 (LWP 2365) exited]
> > [Thread 0x7f104abf3700 (LWP 2371) exited]
> > [Thread 0x7f104a9f2700 (LWP 2372) exited]
> > [Thread 0x7f104a7f1700 (LWP 2373) exited]
> > [Thread 0x7f104a5f0700 (LWP 2374) exited]
> > [Inferior 1 (process 2351) exited with code 0177]
> >
> >
> > Any ideas on what could be the culprit?
> >
> > Thank you!
> > Rares
> >
>
Re: [C++] parquet::arrow::WriteTable process exited with code 0177
Posted by Micah Kornfield <em...@gmail.com>.
Hi Rares,
Based on the stack trace, I'm not seeing any Arrow/Parquet related frames.
Two things come to mind:
1. This seems like it might be heap corruption (see
https://stackoverflow.com/questions/10578315/segmentation-fault-in-dl-runtime-resolve).
Try running using ASAN or valgrind.
2. (Less likely) It could have something to do with whether exceptions
have been compiling the parquet library (Arrow catches exceptions and
translates them to statuses).
-Micah
On Sat, May 21, 2022 at 9:55 PM Rares Vernica <rv...@gmail.com> wrote:
> Hello,
>
> We have a plugin that writes both Arrow and Parquet format files. We are
> experiencing an issue when using Parquet format, while Arrow format works
> just fine. More exactly, the process crashes in parquet::arrow::WriteTable.
>
> Using gdb we identified the line when the process crashes
>
> https://github.com/Paradigm4/bridge/blob/8c34f3b940b10a38eb8900cd05d7ee6aa7c082b0/src/PhysicalXSave.cpp#L494
>
> The gdb output is:
>
> (gdb) catch syscall exit_group
> Catchpoint 1 (syscall 'exit_group' [231])
> (gdb) c
> Continuing.
> [Switching to Thread 0x7f104b7f6700 (LWP 2363)]
>
> Catchpoint 1 (call to syscall exit_group), 0x00007f105c031901 in _exit ()
> from /lib64/ld-linux-x86-64.so.2
> (gdb) bt
> #0 0x00007f105c031901 in _exit () from /lib64/ld-linux-x86-64.so.2
> #1 0x00007f105c0276ba in _dl_signal_error () from
> /lib64/ld-linux-x86-64.so.2
> #2 0x00007f105c02776e in _dl_signal_cerror () from
> /lib64/ld-linux-x86-64.so.2
> #3 0x00007f105c0229aa in _dl_lookup_symbol_x () from
> /lib64/ld-linux-x86-64.so.2
> #4 0x00007f105c026dee in _dl_fixup () from /lib64/ld-linux-x86-64.so.2
> #5 0x00007f105c02eaaa in _dl_runtime_resolve_xsavec () from
> /lib64/ld-linux-x86-64.so.2
> #6 0x00007f1051926a88 in scidb::ArrowWriter::finalize
> (this=this@entry=0x7f104b7f2e40,
> arrowBuffer=std::shared_ptr (empty) 0x0) at PhysicalXSave.cpp:494
> #7 0x00007f105192802f in scidb::ArrowWriter::writeArrowBuffer
> (this=this@entry=0x7f104b7f2e40, chunkIters=std::vector of length 1,
> capacity 1 = {...}, coordMapper=...,
> arrowBuffer=std::shared_ptr (empty) 0x0) at PhysicalXSave.cpp:280
> #8 0x00007f105192b35e in scidb::PhysicalXSave::execute
> (this=0x7f1034004f00, inputArrays=..., query=warning: RTTI symbol not found
> for class 'std::_Sp_counted_ptr_inplace<scidb::Query,
> std::allocator<scidb::Query>, (__gnu_cxx::_Lock_policy)2>'
> warning: RTTI symbol not found for class
> 'std::_Sp_counted_ptr_inplace<scidb::Query, std::allocator<scidb::Query>,
> (__gnu_cxx::_Lock_policy)2>'
> std::shared_ptr (count 8, weak 8) 0x8f348e0) at PhysicalXSave.cpp:879
> #9 0x0000000000af71b7 in
>
> scidb::PhysicalOperator::executeWrapper(std::vector<std::shared_ptr<scidb::Array>,
> std::allocator<std::shared_ptr<scidb::Array> > >&,
> std::shared_ptr<scidb::Query>) ()
> #10 0x0000000000b3bc3d in
>
> scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::PhysicalQueryPlanNode>
> const&, std::shared_ptr<scidb::Query> const&, int) ()
> #11 0x0000000000b3c1b5 in
> scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::Query> const&) ()
> #12 0x0000000000a0dd49 in
> scidb::ServerMessageHandleJob::handleExecutePhysicalPlan() ()
> #13 0x0000000000a0ee23 in
> scidb::ServerMessageHandleJob::handlePreparePhysicalPlan() ()
> #14 0x0000000000a0addf in scidb::ServerMessageHandleJob::run() ()
> #15 0x0000000000a32ed8 in
> scidb::Job::executeOnQueue(std::weak_ptr<scidb::WorkQueue>&,
> std::shared_ptr<scidb::SerializationCtx>&) ()
> #16 0x0000000000a6e938 in scidb::WorkQueue::WorkQueueJob::run() ()
> #17 0x0000000000a32ff0 in scidb::Job::execute() ()
> #18 0x0000000000a3fefc in scidb::Thread::_threadFunction() ()
> #19 0x0000000000a409b9 in scidb::Thread::threadFunction(void*) ()
> #20 0x00007f1058fc8ea5 in start_thread () from /lib64/libpthread.so.0
> #21 0x00007f1057ea7b0d in clone () from /lib64/libc.so.6
> (gdb) c
> Continuing.
> [Thread 0x7f10525ff700 (LWP 2355) exited]
> [Thread 0x7f104bbf8700 (LWP 2361) exited]
> [Thread 0x7f104b9f7700 (LWP 2362) exited]
> [Thread 0x7f104b7f6700 (LWP 2363) exited]
> [Thread 0x7f104b5f5700 (LWP 2364) exited]
> [Thread 0x7f104b3f4700 (LWP 2365) exited]
> [Thread 0x7f104abf3700 (LWP 2371) exited]
> [Thread 0x7f104a9f2700 (LWP 2372) exited]
> [Thread 0x7f104a7f1700 (LWP 2373) exited]
> [Thread 0x7f104a5f0700 (LWP 2374) exited]
> [Inferior 1 (process 2351) exited with code 0177]
>
>
> Any ideas on what could be the culprit?
>
> Thank you!
> Rares
>