You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Rares Vernica <rv...@gmail.com> on 2018/04/02 01:48:34 UTC
Upstream segmentation fault when using 0.9.0, OK with 0.8.0
Hello,
I'm using libarrow.so to build a simplified SciDB "plugin". The plugin gets
loaded dynamically into a running SciDB instance. The plugin just does
arrow::default_memory_pool(). With Arrow 0.8.0, the plugin gets loaded
successfully and can be used in SciDB. With Arrow 0.9.0, SciDB crashes when
it tires to load the .so file of the plugin. I'm not sure if it is an Arrow
issue or some non-Arrow issue that just pops up now.
The example used here is highly simplified. I'm experiencing this situation
in the more complex accelerated_io_tools SciDB plugin
https://github.com/Paradigm4/accelerated_io_tools I've been using this
plugin for a while successfully with Arrow 0.8.0.
I'm using a Ubuntu Trusty instance with Arrow packages from
red-data-tools.org. I also tried with Arrow packages compiled from source
using g++ 4.9, but the issue is still present.
I compile the plugin like this:
"/usr/bin/g++-4.9" -W -Wextra -Wall -Wno-unused-parameter
-Wno-variadic-macros -Wno-strict-aliasing -Wno-long-long -Wno-unused -fPIC
-D_STDC_FORMAT_MACROS -Wno-system-headers -g -DNDEBUG -D_STDC_LIMIT_MACROS
-fno-omit-frame-pointer -std=c++14 -DCPP11 -I.
-DPROJECT_ROOT="\"/opt/scidb/18.1\""
-I"/opt/scidb/18.1/3rdparty/boost/include/" -I"/opt/scidb/18.1/include" -o
liblimit.so plugin.cpp LogicalLimit.cpp PhysicalLimit.cpp -shared
-Wl,-soname,liblimit.so -L. -L"/opt/scidb/18.1/3rdparty/boost/lib"
-L"/opt/scidb/18.1/lib" -Wl,-rpath,/opt/scidb/18.1/lib: -lm -larrow
Here is the backtrace when I try to load the plugin:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fcf89669700 (LWP 1096)]
0x00007fcf96048240 in std::string::resize(unsigned long, char) ()
from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0 0x00007fcf96048240 in std::string::resize(unsigned long, char) ()
from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#1 0x00007fcf985081e8 in
google::protobuf::internal::WireFormatLite::ReadString(google::protobuf::io::CodedInputStream*,
std::string*) ()
from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
#2 0x00007fcf98545539 in
google::protobuf::FileDescriptorProto::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*)
()
from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
#3 0x00007fcf8859b11c in ?? () from /usr/lib/x86_64-linux-gnu/libarrow.so.0
#4 0x00007fcf885e1533 in ?? () from /usr/lib/x86_64-linux-gnu/libarrow.so.0
#5 0x00007fcf885a278a in ?? () from /usr/lib/x86_64-linux-gnu/libarrow.so.0
#6 0x00007fcf885808ac in ?? () from /usr/lib/x86_64-linux-gnu/libarrow.so.0
#7 0x00007fcf998081da in ?? () from /lib64/ld-linux-x86-64.so.2
#8 0x00007fcf998082c3 in ?? () from /lib64/ld-linux-x86-64.so.2
#9 0x00007fcf9980cd00 in ?? () from /lib64/ld-linux-x86-64.so.2
#10 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2
#11 0x00007fcf9980c44b in ?? () from /lib64/ld-linux-x86-64.so.2
#12 0x00007fcf991cf02b in ?? () from /lib/x86_64-linux-gnu/libdl.so.2
#13 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2
#14 0x00007fcf991cf62d in ?? () from /lib/x86_64-linux-gnu/libdl.so.2
#15 0x00007fcf991cf0c1 in dlopen () from /lib/x86_64-linux-gnu/libdl.so.2
#16 0x0000000000951c8b in scidb::PluginManager::findModule(std::string
const&, bool*) ()
#17 0x000000000095235f in scidb::PluginManager::loadLibrary(std::string
const&, bool) ()
#18 0x0000000000ccb1a2 in
scidb::PhysicalLoadLibrary::execute(std::vector<std::shared_ptr<scidb::Array>,
std::allocator<std::shared_ptr<scidb::Array> > >&,
std::shared_ptr<scidb::Query>) ()
#19 0x0000000000a37c6f in
scidb::PhysicalOperator::executeWrapper(std::vector<std::shared_ptr<scidb::Array>,
std::allocator<std::shared_ptr<scidb::Array> > >&,
std::shared_ptr<scidb::Query>) ()
#20 0x00000000009e5628 in
scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::PhysicalQueryPlanNode>,
std::shared_ptr<scidb::Query>, int) ()
#21 0x00000000009e5dd9 in
scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::Query>) ()
#22 0x0000000000a1c6f4 in
scidb::SciDBExecutor::completeExecuteQuery(scidb::QueryResult&,
std::shared_ptr<scidb::Query> const&) ()
#23 0x0000000000903964 in
scidb::ClientMessageHandleJob::completeExecuteQuery(std::shared_ptr<scidb::QueryResult>
const&) ()
#24 0x00000000009005a6 in scidb::ClientMessageHandleJob::run() ()
#25 0x000000000093d59a in
scidb::Job::executeOnQueue(std::weak_ptr<scidb::WorkQueue>&,
std::shared_ptr<scidb::SerializationCtx>&) ()
#26 0x00000000008b34b7 in
scidb::WorkQueue::invokeWithContext(boost::function<void
(std::weak_ptr<scidb::WorkQueue>&,
std::shared_ptr<scidb::SerializationCtx>&)>&,
std::shared_ptr<scidb::SerializationCtx>&,
std::weak_ptr<scidb::WorkQueue>&) ()
#27 0x0000000000972ac3 in scidb::WorkQueue::WorkQueueJob::run() ()
#28 0x000000000093d6d1 in scidb::Job::execute() ()
#29 0x0000000000947f39 in scidb::Thread::_threadFunction() ()
#30 0x0000000000948b39 in scidb::Thread::threadFunction(void*) ()
#31 0x00007fcf993da184 in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#32 0x00007fcf9579a03d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Notice libarrow in frames #3-6. Also interesting is libprotobuf in frames
#1-2.
Any ideas are welcome.
Thanks!
Rares
Re: Upstream segmentation fault when using 0.9.0, OK with 0.8.0
Posted by Rares Vernica <rv...@gmail.com>.
Just to confirm, SciDB is using Protocol Buffers. The current version of
SciDB, 18.1, depends on protobuf8.
On Mon, Apr 2, 2018 at 10:11 PM, Uwe L. Korn <uw...@xhochy.com> wrote:
> The problem is that with ORC on, we link to a self-build protobuf version
> but on load in SciDB, it picks up the system one. We actually should build
> the packages with the system one: https://issues.apache.org/
> jira/browse/ARROW-2383
>
> On Tue, Apr 3, 2018, at 7:06 AM, Rares Vernica wrote:
> > Hi Wes,
> >
> > That did it! Thanks so much for the pointer.
> >
> > BTW, enjoy your time off.
> >
> > Cheers,
> > Rares
> >
> > On Mon, Apr 2, 2018 at 12:33 PM, Wes McKinney <we...@gmail.com>
> wrote:
> >
> > > hi Rares -- does SciDB use Protocol Buffers? You may be running into a
> > > version conflict with the vendored protocol buffers libraries that's
> > > part of -DARROW_ORC=on
> > >
> > > Wes
> > >
> > > On Sun, Apr 1, 2018 at 9:48 PM, Rares Vernica <rv...@gmail.com>
> wrote:
> > > > Hello,
> > > >
> > > > I'm using libarrow.so to build a simplified SciDB "plugin". The
> plugin
> > > gets
> > > > loaded dynamically into a running SciDB instance. The plugin just
> does
> > > > arrow::default_memory_pool(). With Arrow 0.8.0, the plugin gets
> loaded
> > > > successfully and can be used in SciDB. With Arrow 0.9.0, SciDB
> crashes
> > > when
> > > > it tires to load the .so file of the plugin. I'm not sure if it is an
> > > Arrow
> > > > issue or some non-Arrow issue that just pops up now.
> > > >
> > > > The example used here is highly simplified. I'm experiencing this
> > > situation
> > > > in the more complex accelerated_io_tools SciDB plugin
> > > > https://github.com/Paradigm4/accelerated_io_tools I've been using
> this
> > > > plugin for a while successfully with Arrow 0.8.0.
> > > >
> > > > I'm using a Ubuntu Trusty instance with Arrow packages from
> > > > red-data-tools.org. I also tried with Arrow packages compiled from
> > > source
> > > > using g++ 4.9, but the issue is still present.
> > > >
> > > > I compile the plugin like this:
> > > >
> > > > "/usr/bin/g++-4.9" -W -Wextra -Wall -Wno-unused-parameter
> > > > -Wno-variadic-macros -Wno-strict-aliasing -Wno-long-long -Wno-unused
> > > -fPIC
> > > > -D_STDC_FORMAT_MACROS -Wno-system-headers -g -DNDEBUG
> > > -D_STDC_LIMIT_MACROS
> > > > -fno-omit-frame-pointer -std=c++14 -DCPP11 -I.
> > > > -DPROJECT_ROOT="\"/opt/scidb/18.1\""
> > > > -I"/opt/scidb/18.1/3rdparty/boost/include/"
> -I"/opt/scidb/18.1/include"
> > > -o
> > > > liblimit.so plugin.cpp LogicalLimit.cpp PhysicalLimit.cpp -shared
> > > > -Wl,-soname,liblimit.so -L. -L"/opt/scidb/18.1/3rdparty/boost/lib"
> > > > -L"/opt/scidb/18.1/lib" -Wl,-rpath,/opt/scidb/18.1/lib: -lm -larrow
> > > >
> > > > Here is the backtrace when I try to load the plugin:
> > > >
> > > > Program received signal SIGSEGV, Segmentation fault.
> > > > [Switching to Thread 0x7fcf89669700 (LWP 1096)]
> > > > 0x00007fcf96048240 in std::string::resize(unsigned long, char) ()
> > > > from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> > > > (gdb) bt
> > > > #0 0x00007fcf96048240 in std::string::resize(unsigned long, char) ()
> > > > from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> > > > #1 0x00007fcf985081e8 in
> > > > google::protobuf::internal::WireFormatLite::ReadString(
> > > google::protobuf::io::CodedInputStream*,
> > > > std::string*) ()
> > > > from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
> > > > #2 0x00007fcf98545539 in
> > > > google::protobuf::FileDescriptorProto::MergePartialFromCodedStream(
> > > google::protobuf::io::CodedInputStream*)
> > > > ()
> > > > from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
> > > > #3 0x00007fcf8859b11c in ?? () from /usr/lib/x86_64-linux-gnu/
> > > libarrow.so.0
> > > > #4 0x00007fcf885e1533 in ?? () from /usr/lib/x86_64-linux-gnu/
> > > libarrow.so.0
> > > > #5 0x00007fcf885a278a in ?? () from /usr/lib/x86_64-linux-gnu/
> > > libarrow.so.0
> > > > #6 0x00007fcf885808ac in ?? () from /usr/lib/x86_64-linux-gnu/
> > > libarrow.so.0
> > > > #7 0x00007fcf998081da in ?? () from /lib64/ld-linux-x86-64.so.2
> > > > #8 0x00007fcf998082c3 in ?? () from /lib64/ld-linux-x86-64.so.2
> > > > #9 0x00007fcf9980cd00 in ?? () from /lib64/ld-linux-x86-64.so.2
> > > > #10 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2
> > > > #11 0x00007fcf9980c44b in ?? () from /lib64/ld-linux-x86-64.so.2
> > > > #12 0x00007fcf991cf02b in ?? () from /lib/x86_64-linux-gnu/libdl.
> so.2
> > > > #13 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2
> > > > #14 0x00007fcf991cf62d in ?? () from /lib/x86_64-linux-gnu/libdl.
> so.2
> > > > #15 0x00007fcf991cf0c1 in dlopen () from /lib/x86_64-linux-gnu/libdl.
> > > so.2
> > > > #16 0x0000000000951c8b in scidb::PluginManager::
> findModule(std::string
> > > > const&, bool*) ()
> > > > #17 0x000000000095235f in scidb::PluginManager::
> loadLibrary(std::string
> > > > const&, bool) ()
> > > > #18 0x0000000000ccb1a2 in
> > > > scidb::PhysicalLoadLibrary::execute(std::vector<std::
> > > shared_ptr<scidb::Array>,
> > > > std::allocator<std::shared_ptr<scidb::Array> > >&,
> > > > std::shared_ptr<scidb::Query>) ()
> > > > #19 0x0000000000a37c6f in
> > > > scidb::PhysicalOperator::executeWrapper(std::vector<
> > > std::shared_ptr<scidb::Array>,
> > > > std::allocator<std::shared_ptr<scidb::Array> > >&,
> > > > std::shared_ptr<scidb::Query>) ()
> > > > #20 0x00000000009e5628 in
> > > > scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb:
> > > :PhysicalQueryPlanNode>,
> > > > std::shared_ptr<scidb::Query>, int) ()
> > > > #21 0x00000000009e5dd9 in
> > > > scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::Query>) ()
> > > > #22 0x0000000000a1c6f4 in
> > > > scidb::SciDBExecutor::completeExecuteQuery(scidb::QueryResult&,
> > > > std::shared_ptr<scidb::Query> const&) ()
> > > > #23 0x0000000000903964 in
> > > > scidb::ClientMessageHandleJob::completeExecuteQuery(std::
> > > shared_ptr<scidb::QueryResult>
> > > > const&) ()
> > > > #24 0x00000000009005a6 in scidb::ClientMessageHandleJob::run() ()
> > > > #25 0x000000000093d59a in
> > > > scidb::Job::executeOnQueue(std::weak_ptr<scidb::WorkQueue>&,
> > > > std::shared_ptr<scidb::SerializationCtx>&) ()
> > > > #26 0x00000000008b34b7 in
> > > > scidb::WorkQueue::invokeWithContext(boost::function<void
> > > > (std::weak_ptr<scidb::WorkQueue>&,
> > > > std::shared_ptr<scidb::SerializationCtx>&)>&,
> > > > std::shared_ptr<scidb::SerializationCtx>&,
> > > > std::weak_ptr<scidb::WorkQueue>&) ()
> > > > #27 0x0000000000972ac3 in scidb::WorkQueue::WorkQueueJob::run() ()
> > > > #28 0x000000000093d6d1 in scidb::Job::execute() ()
> > > > #29 0x0000000000947f39 in scidb::Thread::_threadFunction() ()
> > > > #30 0x0000000000948b39 in scidb::Thread::threadFunction(void*) ()
> > > > #31 0x00007fcf993da184 in start_thread ()
> > > > from /lib/x86_64-linux-gnu/libpthread.so.0
> > > > #32 0x00007fcf9579a03d in clone () from
> /lib/x86_64-linux-gnu/libc.so.6
> > > >
> > > > Notice libarrow in frames #3-6. Also interesting is libprotobuf in
> frames
> > > > #1-2.
> > > >
> > > > Any ideas are welcome.
> > > >
> > > > Thanks!
> > > > Rares
> > >
>
Re: Upstream segmentation fault when using 0.9.0, OK with 0.8.0
Posted by "Uwe L. Korn" <uw...@xhochy.com>.
The problem is that with ORC on, we link to a self-build protobuf version but on load in SciDB, it picks up the system one. We actually should build the packages with the system one: https://issues.apache.org/jira/browse/ARROW-2383
On Tue, Apr 3, 2018, at 7:06 AM, Rares Vernica wrote:
> Hi Wes,
>
> That did it! Thanks so much for the pointer.
>
> BTW, enjoy your time off.
>
> Cheers,
> Rares
>
> On Mon, Apr 2, 2018 at 12:33 PM, Wes McKinney <we...@gmail.com> wrote:
>
> > hi Rares -- does SciDB use Protocol Buffers? You may be running into a
> > version conflict with the vendored protocol buffers libraries that's
> > part of -DARROW_ORC=on
> >
> > Wes
> >
> > On Sun, Apr 1, 2018 at 9:48 PM, Rares Vernica <rv...@gmail.com> wrote:
> > > Hello,
> > >
> > > I'm using libarrow.so to build a simplified SciDB "plugin". The plugin
> > gets
> > > loaded dynamically into a running SciDB instance. The plugin just does
> > > arrow::default_memory_pool(). With Arrow 0.8.0, the plugin gets loaded
> > > successfully and can be used in SciDB. With Arrow 0.9.0, SciDB crashes
> > when
> > > it tires to load the .so file of the plugin. I'm not sure if it is an
> > Arrow
> > > issue or some non-Arrow issue that just pops up now.
> > >
> > > The example used here is highly simplified. I'm experiencing this
> > situation
> > > in the more complex accelerated_io_tools SciDB plugin
> > > https://github.com/Paradigm4/accelerated_io_tools I've been using this
> > > plugin for a while successfully with Arrow 0.8.0.
> > >
> > > I'm using a Ubuntu Trusty instance with Arrow packages from
> > > red-data-tools.org. I also tried with Arrow packages compiled from
> > source
> > > using g++ 4.9, but the issue is still present.
> > >
> > > I compile the plugin like this:
> > >
> > > "/usr/bin/g++-4.9" -W -Wextra -Wall -Wno-unused-parameter
> > > -Wno-variadic-macros -Wno-strict-aliasing -Wno-long-long -Wno-unused
> > -fPIC
> > > -D_STDC_FORMAT_MACROS -Wno-system-headers -g -DNDEBUG
> > -D_STDC_LIMIT_MACROS
> > > -fno-omit-frame-pointer -std=c++14 -DCPP11 -I.
> > > -DPROJECT_ROOT="\"/opt/scidb/18.1\""
> > > -I"/opt/scidb/18.1/3rdparty/boost/include/" -I"/opt/scidb/18.1/include"
> > -o
> > > liblimit.so plugin.cpp LogicalLimit.cpp PhysicalLimit.cpp -shared
> > > -Wl,-soname,liblimit.so -L. -L"/opt/scidb/18.1/3rdparty/boost/lib"
> > > -L"/opt/scidb/18.1/lib" -Wl,-rpath,/opt/scidb/18.1/lib: -lm -larrow
> > >
> > > Here is the backtrace when I try to load the plugin:
> > >
> > > Program received signal SIGSEGV, Segmentation fault.
> > > [Switching to Thread 0x7fcf89669700 (LWP 1096)]
> > > 0x00007fcf96048240 in std::string::resize(unsigned long, char) ()
> > > from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> > > (gdb) bt
> > > #0 0x00007fcf96048240 in std::string::resize(unsigned long, char) ()
> > > from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> > > #1 0x00007fcf985081e8 in
> > > google::protobuf::internal::WireFormatLite::ReadString(
> > google::protobuf::io::CodedInputStream*,
> > > std::string*) ()
> > > from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
> > > #2 0x00007fcf98545539 in
> > > google::protobuf::FileDescriptorProto::MergePartialFromCodedStream(
> > google::protobuf::io::CodedInputStream*)
> > > ()
> > > from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
> > > #3 0x00007fcf8859b11c in ?? () from /usr/lib/x86_64-linux-gnu/
> > libarrow.so.0
> > > #4 0x00007fcf885e1533 in ?? () from /usr/lib/x86_64-linux-gnu/
> > libarrow.so.0
> > > #5 0x00007fcf885a278a in ?? () from /usr/lib/x86_64-linux-gnu/
> > libarrow.so.0
> > > #6 0x00007fcf885808ac in ?? () from /usr/lib/x86_64-linux-gnu/
> > libarrow.so.0
> > > #7 0x00007fcf998081da in ?? () from /lib64/ld-linux-x86-64.so.2
> > > #8 0x00007fcf998082c3 in ?? () from /lib64/ld-linux-x86-64.so.2
> > > #9 0x00007fcf9980cd00 in ?? () from /lib64/ld-linux-x86-64.so.2
> > > #10 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2
> > > #11 0x00007fcf9980c44b in ?? () from /lib64/ld-linux-x86-64.so.2
> > > #12 0x00007fcf991cf02b in ?? () from /lib/x86_64-linux-gnu/libdl.so.2
> > > #13 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2
> > > #14 0x00007fcf991cf62d in ?? () from /lib/x86_64-linux-gnu/libdl.so.2
> > > #15 0x00007fcf991cf0c1 in dlopen () from /lib/x86_64-linux-gnu/libdl.
> > so.2
> > > #16 0x0000000000951c8b in scidb::PluginManager::findModule(std::string
> > > const&, bool*) ()
> > > #17 0x000000000095235f in scidb::PluginManager::loadLibrary(std::string
> > > const&, bool) ()
> > > #18 0x0000000000ccb1a2 in
> > > scidb::PhysicalLoadLibrary::execute(std::vector<std::
> > shared_ptr<scidb::Array>,
> > > std::allocator<std::shared_ptr<scidb::Array> > >&,
> > > std::shared_ptr<scidb::Query>) ()
> > > #19 0x0000000000a37c6f in
> > > scidb::PhysicalOperator::executeWrapper(std::vector<
> > std::shared_ptr<scidb::Array>,
> > > std::allocator<std::shared_ptr<scidb::Array> > >&,
> > > std::shared_ptr<scidb::Query>) ()
> > > #20 0x00000000009e5628 in
> > > scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb:
> > :PhysicalQueryPlanNode>,
> > > std::shared_ptr<scidb::Query>, int) ()
> > > #21 0x00000000009e5dd9 in
> > > scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::Query>) ()
> > > #22 0x0000000000a1c6f4 in
> > > scidb::SciDBExecutor::completeExecuteQuery(scidb::QueryResult&,
> > > std::shared_ptr<scidb::Query> const&) ()
> > > #23 0x0000000000903964 in
> > > scidb::ClientMessageHandleJob::completeExecuteQuery(std::
> > shared_ptr<scidb::QueryResult>
> > > const&) ()
> > > #24 0x00000000009005a6 in scidb::ClientMessageHandleJob::run() ()
> > > #25 0x000000000093d59a in
> > > scidb::Job::executeOnQueue(std::weak_ptr<scidb::WorkQueue>&,
> > > std::shared_ptr<scidb::SerializationCtx>&) ()
> > > #26 0x00000000008b34b7 in
> > > scidb::WorkQueue::invokeWithContext(boost::function<void
> > > (std::weak_ptr<scidb::WorkQueue>&,
> > > std::shared_ptr<scidb::SerializationCtx>&)>&,
> > > std::shared_ptr<scidb::SerializationCtx>&,
> > > std::weak_ptr<scidb::WorkQueue>&) ()
> > > #27 0x0000000000972ac3 in scidb::WorkQueue::WorkQueueJob::run() ()
> > > #28 0x000000000093d6d1 in scidb::Job::execute() ()
> > > #29 0x0000000000947f39 in scidb::Thread::_threadFunction() ()
> > > #30 0x0000000000948b39 in scidb::Thread::threadFunction(void*) ()
> > > #31 0x00007fcf993da184 in start_thread ()
> > > from /lib/x86_64-linux-gnu/libpthread.so.0
> > > #32 0x00007fcf9579a03d in clone () from /lib/x86_64-linux-gnu/libc.so.6
> > >
> > > Notice libarrow in frames #3-6. Also interesting is libprotobuf in frames
> > > #1-2.
> > >
> > > Any ideas are welcome.
> > >
> > > Thanks!
> > > Rares
> >
Re: Upstream segmentation fault when using 0.9.0, OK with 0.8.0
Posted by Rares Vernica <rv...@gmail.com>.
Hi Wes,
That did it! Thanks so much for the pointer.
BTW, enjoy your time off.
Cheers,
Rares
On Mon, Apr 2, 2018 at 12:33 PM, Wes McKinney <we...@gmail.com> wrote:
> hi Rares -- does SciDB use Protocol Buffers? You may be running into a
> version conflict with the vendored protocol buffers libraries that's
> part of -DARROW_ORC=on
>
> Wes
>
> On Sun, Apr 1, 2018 at 9:48 PM, Rares Vernica <rv...@gmail.com> wrote:
> > Hello,
> >
> > I'm using libarrow.so to build a simplified SciDB "plugin". The plugin
> gets
> > loaded dynamically into a running SciDB instance. The plugin just does
> > arrow::default_memory_pool(). With Arrow 0.8.0, the plugin gets loaded
> > successfully and can be used in SciDB. With Arrow 0.9.0, SciDB crashes
> when
> > it tires to load the .so file of the plugin. I'm not sure if it is an
> Arrow
> > issue or some non-Arrow issue that just pops up now.
> >
> > The example used here is highly simplified. I'm experiencing this
> situation
> > in the more complex accelerated_io_tools SciDB plugin
> > https://github.com/Paradigm4/accelerated_io_tools I've been using this
> > plugin for a while successfully with Arrow 0.8.0.
> >
> > I'm using a Ubuntu Trusty instance with Arrow packages from
> > red-data-tools.org. I also tried with Arrow packages compiled from
> source
> > using g++ 4.9, but the issue is still present.
> >
> > I compile the plugin like this:
> >
> > "/usr/bin/g++-4.9" -W -Wextra -Wall -Wno-unused-parameter
> > -Wno-variadic-macros -Wno-strict-aliasing -Wno-long-long -Wno-unused
> -fPIC
> > -D_STDC_FORMAT_MACROS -Wno-system-headers -g -DNDEBUG
> -D_STDC_LIMIT_MACROS
> > -fno-omit-frame-pointer -std=c++14 -DCPP11 -I.
> > -DPROJECT_ROOT="\"/opt/scidb/18.1\""
> > -I"/opt/scidb/18.1/3rdparty/boost/include/" -I"/opt/scidb/18.1/include"
> -o
> > liblimit.so plugin.cpp LogicalLimit.cpp PhysicalLimit.cpp -shared
> > -Wl,-soname,liblimit.so -L. -L"/opt/scidb/18.1/3rdparty/boost/lib"
> > -L"/opt/scidb/18.1/lib" -Wl,-rpath,/opt/scidb/18.1/lib: -lm -larrow
> >
> > Here is the backtrace when I try to load the plugin:
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > [Switching to Thread 0x7fcf89669700 (LWP 1096)]
> > 0x00007fcf96048240 in std::string::resize(unsigned long, char) ()
> > from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> > (gdb) bt
> > #0 0x00007fcf96048240 in std::string::resize(unsigned long, char) ()
> > from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> > #1 0x00007fcf985081e8 in
> > google::protobuf::internal::WireFormatLite::ReadString(
> google::protobuf::io::CodedInputStream*,
> > std::string*) ()
> > from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
> > #2 0x00007fcf98545539 in
> > google::protobuf::FileDescriptorProto::MergePartialFromCodedStream(
> google::protobuf::io::CodedInputStream*)
> > ()
> > from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
> > #3 0x00007fcf8859b11c in ?? () from /usr/lib/x86_64-linux-gnu/
> libarrow.so.0
> > #4 0x00007fcf885e1533 in ?? () from /usr/lib/x86_64-linux-gnu/
> libarrow.so.0
> > #5 0x00007fcf885a278a in ?? () from /usr/lib/x86_64-linux-gnu/
> libarrow.so.0
> > #6 0x00007fcf885808ac in ?? () from /usr/lib/x86_64-linux-gnu/
> libarrow.so.0
> > #7 0x00007fcf998081da in ?? () from /lib64/ld-linux-x86-64.so.2
> > #8 0x00007fcf998082c3 in ?? () from /lib64/ld-linux-x86-64.so.2
> > #9 0x00007fcf9980cd00 in ?? () from /lib64/ld-linux-x86-64.so.2
> > #10 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2
> > #11 0x00007fcf9980c44b in ?? () from /lib64/ld-linux-x86-64.so.2
> > #12 0x00007fcf991cf02b in ?? () from /lib/x86_64-linux-gnu/libdl.so.2
> > #13 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2
> > #14 0x00007fcf991cf62d in ?? () from /lib/x86_64-linux-gnu/libdl.so.2
> > #15 0x00007fcf991cf0c1 in dlopen () from /lib/x86_64-linux-gnu/libdl.
> so.2
> > #16 0x0000000000951c8b in scidb::PluginManager::findModule(std::string
> > const&, bool*) ()
> > #17 0x000000000095235f in scidb::PluginManager::loadLibrary(std::string
> > const&, bool) ()
> > #18 0x0000000000ccb1a2 in
> > scidb::PhysicalLoadLibrary::execute(std::vector<std::
> shared_ptr<scidb::Array>,
> > std::allocator<std::shared_ptr<scidb::Array> > >&,
> > std::shared_ptr<scidb::Query>) ()
> > #19 0x0000000000a37c6f in
> > scidb::PhysicalOperator::executeWrapper(std::vector<
> std::shared_ptr<scidb::Array>,
> > std::allocator<std::shared_ptr<scidb::Array> > >&,
> > std::shared_ptr<scidb::Query>) ()
> > #20 0x00000000009e5628 in
> > scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb:
> :PhysicalQueryPlanNode>,
> > std::shared_ptr<scidb::Query>, int) ()
> > #21 0x00000000009e5dd9 in
> > scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::Query>) ()
> > #22 0x0000000000a1c6f4 in
> > scidb::SciDBExecutor::completeExecuteQuery(scidb::QueryResult&,
> > std::shared_ptr<scidb::Query> const&) ()
> > #23 0x0000000000903964 in
> > scidb::ClientMessageHandleJob::completeExecuteQuery(std::
> shared_ptr<scidb::QueryResult>
> > const&) ()
> > #24 0x00000000009005a6 in scidb::ClientMessageHandleJob::run() ()
> > #25 0x000000000093d59a in
> > scidb::Job::executeOnQueue(std::weak_ptr<scidb::WorkQueue>&,
> > std::shared_ptr<scidb::SerializationCtx>&) ()
> > #26 0x00000000008b34b7 in
> > scidb::WorkQueue::invokeWithContext(boost::function<void
> > (std::weak_ptr<scidb::WorkQueue>&,
> > std::shared_ptr<scidb::SerializationCtx>&)>&,
> > std::shared_ptr<scidb::SerializationCtx>&,
> > std::weak_ptr<scidb::WorkQueue>&) ()
> > #27 0x0000000000972ac3 in scidb::WorkQueue::WorkQueueJob::run() ()
> > #28 0x000000000093d6d1 in scidb::Job::execute() ()
> > #29 0x0000000000947f39 in scidb::Thread::_threadFunction() ()
> > #30 0x0000000000948b39 in scidb::Thread::threadFunction(void*) ()
> > #31 0x00007fcf993da184 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #32 0x00007fcf9579a03d in clone () from /lib/x86_64-linux-gnu/libc.so.6
> >
> > Notice libarrow in frames #3-6. Also interesting is libprotobuf in frames
> > #1-2.
> >
> > Any ideas are welcome.
> >
> > Thanks!
> > Rares
>
Re: Upstream segmentation fault when using 0.9.0, OK with 0.8.0
Posted by Wes McKinney <we...@gmail.com>.
hi Rares -- does SciDB use Protocol Buffers? You may be running into a
version conflict with the vendored protocol buffers libraries that's
part of -DARROW_ORC=on
Wes
On Sun, Apr 1, 2018 at 9:48 PM, Rares Vernica <rv...@gmail.com> wrote:
> Hello,
>
> I'm using libarrow.so to build a simplified SciDB "plugin". The plugin gets
> loaded dynamically into a running SciDB instance. The plugin just does
> arrow::default_memory_pool(). With Arrow 0.8.0, the plugin gets loaded
> successfully and can be used in SciDB. With Arrow 0.9.0, SciDB crashes when
> it tires to load the .so file of the plugin. I'm not sure if it is an Arrow
> issue or some non-Arrow issue that just pops up now.
>
> The example used here is highly simplified. I'm experiencing this situation
> in the more complex accelerated_io_tools SciDB plugin
> https://github.com/Paradigm4/accelerated_io_tools I've been using this
> plugin for a while successfully with Arrow 0.8.0.
>
> I'm using a Ubuntu Trusty instance with Arrow packages from
> red-data-tools.org. I also tried with Arrow packages compiled from source
> using g++ 4.9, but the issue is still present.
>
> I compile the plugin like this:
>
> "/usr/bin/g++-4.9" -W -Wextra -Wall -Wno-unused-parameter
> -Wno-variadic-macros -Wno-strict-aliasing -Wno-long-long -Wno-unused -fPIC
> -D_STDC_FORMAT_MACROS -Wno-system-headers -g -DNDEBUG -D_STDC_LIMIT_MACROS
> -fno-omit-frame-pointer -std=c++14 -DCPP11 -I.
> -DPROJECT_ROOT="\"/opt/scidb/18.1\""
> -I"/opt/scidb/18.1/3rdparty/boost/include/" -I"/opt/scidb/18.1/include" -o
> liblimit.so plugin.cpp LogicalLimit.cpp PhysicalLimit.cpp -shared
> -Wl,-soname,liblimit.so -L. -L"/opt/scidb/18.1/3rdparty/boost/lib"
> -L"/opt/scidb/18.1/lib" -Wl,-rpath,/opt/scidb/18.1/lib: -lm -larrow
>
> Here is the backtrace when I try to load the plugin:
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fcf89669700 (LWP 1096)]
> 0x00007fcf96048240 in std::string::resize(unsigned long, char) ()
> from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> (gdb) bt
> #0 0x00007fcf96048240 in std::string::resize(unsigned long, char) ()
> from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> #1 0x00007fcf985081e8 in
> google::protobuf::internal::WireFormatLite::ReadString(google::protobuf::io::CodedInputStream*,
> std::string*) ()
> from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
> #2 0x00007fcf98545539 in
> google::protobuf::FileDescriptorProto::MergePartialFromCodedStream(google::protobuf::io::CodedInputStream*)
> ()
> from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
> #3 0x00007fcf8859b11c in ?? () from /usr/lib/x86_64-linux-gnu/libarrow.so.0
> #4 0x00007fcf885e1533 in ?? () from /usr/lib/x86_64-linux-gnu/libarrow.so.0
> #5 0x00007fcf885a278a in ?? () from /usr/lib/x86_64-linux-gnu/libarrow.so.0
> #6 0x00007fcf885808ac in ?? () from /usr/lib/x86_64-linux-gnu/libarrow.so.0
> #7 0x00007fcf998081da in ?? () from /lib64/ld-linux-x86-64.so.2
> #8 0x00007fcf998082c3 in ?? () from /lib64/ld-linux-x86-64.so.2
> #9 0x00007fcf9980cd00 in ?? () from /lib64/ld-linux-x86-64.so.2
> #10 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2
> #11 0x00007fcf9980c44b in ?? () from /lib64/ld-linux-x86-64.so.2
> #12 0x00007fcf991cf02b in ?? () from /lib/x86_64-linux-gnu/libdl.so.2
> #13 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2
> #14 0x00007fcf991cf62d in ?? () from /lib/x86_64-linux-gnu/libdl.so.2
> #15 0x00007fcf991cf0c1 in dlopen () from /lib/x86_64-linux-gnu/libdl.so.2
> #16 0x0000000000951c8b in scidb::PluginManager::findModule(std::string
> const&, bool*) ()
> #17 0x000000000095235f in scidb::PluginManager::loadLibrary(std::string
> const&, bool) ()
> #18 0x0000000000ccb1a2 in
> scidb::PhysicalLoadLibrary::execute(std::vector<std::shared_ptr<scidb::Array>,
> std::allocator<std::shared_ptr<scidb::Array> > >&,
> std::shared_ptr<scidb::Query>) ()
> #19 0x0000000000a37c6f in
> scidb::PhysicalOperator::executeWrapper(std::vector<std::shared_ptr<scidb::Array>,
> std::allocator<std::shared_ptr<scidb::Array> > >&,
> std::shared_ptr<scidb::Query>) ()
> #20 0x00000000009e5628 in
> scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::PhysicalQueryPlanNode>,
> std::shared_ptr<scidb::Query>, int) ()
> #21 0x00000000009e5dd9 in
> scidb::QueryProcessorImpl::execute(std::shared_ptr<scidb::Query>) ()
> #22 0x0000000000a1c6f4 in
> scidb::SciDBExecutor::completeExecuteQuery(scidb::QueryResult&,
> std::shared_ptr<scidb::Query> const&) ()
> #23 0x0000000000903964 in
> scidb::ClientMessageHandleJob::completeExecuteQuery(std::shared_ptr<scidb::QueryResult>
> const&) ()
> #24 0x00000000009005a6 in scidb::ClientMessageHandleJob::run() ()
> #25 0x000000000093d59a in
> scidb::Job::executeOnQueue(std::weak_ptr<scidb::WorkQueue>&,
> std::shared_ptr<scidb::SerializationCtx>&) ()
> #26 0x00000000008b34b7 in
> scidb::WorkQueue::invokeWithContext(boost::function<void
> (std::weak_ptr<scidb::WorkQueue>&,
> std::shared_ptr<scidb::SerializationCtx>&)>&,
> std::shared_ptr<scidb::SerializationCtx>&,
> std::weak_ptr<scidb::WorkQueue>&) ()
> #27 0x0000000000972ac3 in scidb::WorkQueue::WorkQueueJob::run() ()
> #28 0x000000000093d6d1 in scidb::Job::execute() ()
> #29 0x0000000000947f39 in scidb::Thread::_threadFunction() ()
> #30 0x0000000000948b39 in scidb::Thread::threadFunction(void*) ()
> #31 0x00007fcf993da184 in start_thread ()
> from /lib/x86_64-linux-gnu/libpthread.so.0
> #32 0x00007fcf9579a03d in clone () from /lib/x86_64-linux-gnu/libc.so.6
>
> Notice libarrow in frames #3-6. Also interesting is libprotobuf in frames
> #1-2.
>
> Any ideas are welcome.
>
> Thanks!
> Rares