You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Mostafa Mokhtar (JIRA)" <ji...@apache.org> on 2017/05/25 14:56:04 UTC
[jira] [Resolved] (IMPALA-5302) tcmalloc contention limits CPU
utilization on machines with >40 logical processors
[ https://issues.apache.org/jira/browse/IMPALA-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mostafa Mokhtar resolved IMPALA-5302.
-------------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.9.0
> tcmalloc contention limits CPU utilization on machines with >40 logical processors
> ----------------------------------------------------------------------------------
>
> Key: IMPALA-5302
> URL: https://issues.apache.org/jira/browse/IMPALA-5302
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.8.0
> Reporter: Mostafa Mokhtar
> Assignee: Mostafa Mokhtar
> Priority: Critical
> Labels: memory, perfomance, scalability
> Fix For: Impala 2.9.0
>
> Attachments: Screen Shot 2017-05-10 at 6.44.32 PM.png, thread_stack.txt
>
>
> On modern hardware with 80 logical processors it is very difficult to saturate the CPU on the server, even where there is high concurrency.
> Analysis showed that usage patterns of tcmalloc leads to SpinLock::SlowLock which results in the thread sleeping which results in under utilization of CPU.
> From perf with sampling on sched:sched_switch
> {code}
> Samples: 331K of event 'sched:sched_switch', Event count (approx.): 331431
> Children Self Trace output ◆
> - 0.76% 0.76% impalad:47637 [120] S ==> swapper/22:0 [120] ▒
> base::internal::SpinLockDelay ▒
> system_call_fastpath ▒
> sys_futex ▒
> do_futex ▒
> futex_wait ▒
> futex_wait_queue_me ▒
> schedule ▒
> __schedule ▒
> - 0.76% 0.76% swapper/22:0 [120] R ==> impalad:47637 [120] ▒
> start_secondary ▒
> cpu_startup_entry ▒
> schedule_preempt_disabled ▒
> __schedule ▒
> - 0.73% 0.73% swapper/35:0 [120] R ==> impalad:47588 [120] ▒
> start_secondary ▒
> cpu_startup_entry ▒
> schedule_preempt_disabled ▒
> __schedule ▒
> - 0.73% 0.73% impalad:47588 [120] S ==> swapper/35:0 [120] ▒
> base::internal::SpinLockDelay ▒
> system_call_fastpath ▒
> sys_futex ▒
> do_futex ▒
> futex_wait ▒
> futex_wait_queue_me ▒
> schedule ▒
> __schedule ▒
> Tip: Create an archive with symtabs to analyse on other machine: perf archive
> {code}
> Call stack
> {code}
> CPU Time
> 1 of 101: 27.0% (1.141s of 4.224s)
> impalad ! SpinLock::SpinLoop - [unknown source file]
> impalad ! SpinLock::SlowLock + 0x70 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::Populate + 0x167 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::FetchFromOneSpansSafe + 0x37 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::RemoveRange + 0x63 - [unknown source file]
> impalad ! tcmalloc::ThreadCache::FetchFromCentralCache + 0x62 - [unknown source file]
> impalad ! malloc + 0x27a - [unknown source file]
> impalad ! impala::MemPool::FindChunk + 0x207 - [unknown source file]
> impalad ! impala::RowBatch::ResizeAndAllocateTupleBuffer + 0xa1 - [unknown source file]
> impalad ! impala::HdfsParquetScanner::AssembleRows + 0x7f - [unknown source file]
> impalad ! impala::HdfsParquetScanner::GetNextInternal + 0x277 - [unknown source file]
> impalad ! impala::HdfsParquetScanner::ProcessSplit + 0x73 - [unknown source file]
> impalad ! impala::HdfsScanNode::ProcessSplit + 0x395 - [unknown source file]
> impalad ! impala::HdfsScanNode::ScannerThread + 0xbb8 - [unknown source file]
> impalad ! impala::Thread::SuperviseThread + 0x228 - [unknown source file]
> impalad ! boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void (void)>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run + 0x73 - [unknown source file]
> impalad ! func@0xe4b330 + 0xd9 - [unknown source file]
> libpthread-2.17.so ! start_thread + 0xc4 - [unknown source file]
> libc-2.17.so ! __clone + 0x6c - [unknown source file]
> {code}
> {code}
> CPU Time
> 2 of 101: 21.9% (0.923s of 4.224s)
> impalad ! SpinLock::SpinLoop - [unknown source file]
> impalad ! SpinLock::SlowLock + 0x1d - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::Populate + 0x167 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::FetchFromOneSpansSafe + 0x37 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::RemoveRange + 0x63 - [unknown source file]
> impalad ! tcmalloc::ThreadCache::FetchFromCentralCache + 0x62 - [unknown source file]
> impalad ! malloc + 0x27a - [unknown source file]
> impalad ! impala::MemPool::FindChunk + 0x207 - [unknown source file]
> impalad ! impala::RowBatch::ResizeAndAllocateTupleBuffer + 0xa1 - [unknown source file]
> impalad ! impala::HdfsParquetScanner::AssembleRows + 0x7f - [unknown source file]
> impalad ! impala::HdfsParquetScanner::GetNextInternal + 0x277 - [unknown source file]
> impalad ! impala::HdfsParquetScanner::ProcessSplit + 0x73 - [unknown source file]
> impalad ! impala::HdfsScanNode::ProcessSplit + 0x395 - [unknown source file]
> impalad ! impala::HdfsScanNode::ScannerThread + 0xbb8 - [unknown source file]
> impalad ! impala::Thread::SuperviseThread + 0x228 - [unknown source file]
> impalad ! boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void (void)>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run + 0x73 - [unknown source file]
> impalad ! func@0xe4b330 + 0xd9 - [unknown source file]
> libpthread-2.17.so ! start_thread + 0xc4 - [unknown source file]
> libc-2.17.so ! __clone + 0x6c - [unknown source file]
> {code}
> {code}
> CPU Time
> 3 of 101: 7.1% (0.299s of 4.224s)
> impalad ! SpinLock::SpinLoop - [unknown source file]
> impalad ! SpinLock::SlowLock + 0x70 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::ReleaseToSpans + 0x157 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::ReleaseListToSpans + 0x1a - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::InsertRange + 0x3b - [unknown source file]
> impalad ! tcmalloc::ThreadCache::ReleaseToCentralCache + 0x103 - [unknown source file]
> impalad ! tcmalloc::ThreadCache::ListTooLong + 0x1b - [unknown source file]
> impalad ! operator delete + 0x3bf - [unknown source file]
> impalad ! impala::MemPool::FreeAll + 0x59 - [unknown source file]
> impalad ! impala::RowBatch::Reset + 0x30 - [unknown source file]
> impalad ! impala::PartitionedAggregationNode::GetRowsStreaming + 0x1f6 - [unknown source file]
> impalad ! impala::PartitionedAggregationNode::GetNextInternal + 0x230 - [unknown source file]
> impalad ! impala::PartitionedAggregationNode::GetNext + 0x21 - [unknown source file]
> impalad ! impala::FragmentInstanceState::ExecInternal + 0x187 - [unknown source file]
> impalad ! impala::FragmentInstanceState::Exec + 0x2b8 - [unknown source file]
> impalad ! impala::QueryState::ExecFInstance + 0x1f5 - [unknown source file]
> impalad ! impala::Thread::SuperviseThread + 0x228 - [unknown source file]
> impalad ! boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void (void)>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run + 0x73 - [unknown source file]
> impalad ! func@0xe4b330 + 0xd9 - [unknown source file]
> libpthread-2.17.so ! start_thread + 0xc4 - [unknown source file]
> libc-2.17.so ! __clone + 0x6c - [unknown source file]
> {code}
> {code}
> CPU Time
> 5 of 101: 3.8% (0.162s of 4.224s)
> impalad ! SpinLock::SpinLoop - [unknown source file]
> impalad ! SpinLock::SlowLock + 0x70 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::Populate + 0x167 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::FetchFromOneSpansSafe + 0x37 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::RemoveRange + 0x63 - [unknown source file]
> impalad ! tcmalloc::ThreadCache::FetchFromCentralCache + 0x62 - [unknown source file]
> impalad ! malloc + 0x27a - [unknown source file]
> impalad ! impala::MemPool::FindChunk + 0x353 - [unknown source file]
> impalad ! impala::MemPool::Allocate<(bool)0> + 0x51 - [unknown source file]
> impalad ! impala::RowBatch::RowBatch + 0x183 - [unknown source file]
> impalad ! impala::DataStreamRecvr::SenderQueue::AddBatch + 0x445 - [unknown source file]
> impalad ! impala::DataStreamMgr::AddData + 0x115 - [unknown source file]
> impalad ! impala::ImpalaServer::TransmitData + 0x142 - [unknown source file]
> impalad ! impala::ImpalaInternalServiceProcessor::process_TransmitData + 0x2c0 - [unknown source file]
> impalad ! impala::ImpalaInternalServiceProcessor::dispatchCall + 0xe8 - [unknown source file]
> impalad ! apache::thrift::TDispatchProcessor::process + 0xab - [unknown source file]
> impalad ! apache::thrift::server::TAcceptQueueServer::Task::run + 0x15e - [unknown source file]
> impalad ! impala::ThriftThread::RunRunnable + 0x68 - [unknown source file]
> impalad ! boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>, boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable>>, boost::_bi::value<impala::Promise<unsigned long>*>>>, void>::invoke + 0x61 - [unknown source file]
> impalad ! impala::Thread::SuperviseThread + 0x228 - [unknown source file]
> impalad ! boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void (void)>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run + 0x73 - [unknown source file]
> impalad ! func@0xe4b330 + 0xd9 - [unknown source file]
> libpthread-2.17.so ! start_thread + 0xc4 - [unknown source file]
> libc-2.17.so ! __clone + 0x6c - [unknown source file]
> {code}
> {code}
> CPU Time
> 6 of 101: 3.5% (0.146s of 4.224s)
> impalad ! SpinLock::SpinLoop - [unknown source file]
> impalad ! SpinLock::SlowLock + 0x1d - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::Populate + 0x167 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::FetchFromOneSpansSafe + 0x37 - [unknown source file]
> impalad ! tcmalloc::CentralFreeList::RemoveRange + 0x63 - [unknown source file]
> impalad ! tcmalloc::ThreadCache::FetchFromCentralCache + 0x62 - [unknown source file]
> impalad ! malloc + 0x27a - [unknown source file]
> impalad ! impala::MemPool::FindChunk + 0x353 - [unknown source file]
> impalad ! impala::MemPool::Allocate<(bool)0> + 0x51 - [unknown source file]
> impalad ! impala::RowBatch::RowBatch + 0x183 - [unknown source file]
> impalad ! impala::DataStreamRecvr::SenderQueue::AddBatch + 0x445 - [unknown source file]
> impalad ! impala::DataStreamMgr::AddData + 0x115 - [unknown source file]
> impalad ! impala::ImpalaServer::TransmitData + 0x142 - [unknown source file]
> impalad ! impala::ImpalaInternalServiceProcessor::process_TransmitData + 0x2c0 - [unknown source file]
> impalad ! impala::ImpalaInternalServiceProcessor::dispatchCall + 0xe8 - [unknown source file]
> impalad ! apache::thrift::TDispatchProcessor::process + 0xab - [unknown source file]
> impalad ! apache::thrift::server::TAcceptQueueServer::Task::run + 0x15e - [unknown source file]
> impalad ! impala::ThriftThread::RunRunnable + 0x68 - [unknown source file]
> impalad ! boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>, boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable>>, boost::_bi::value<impala::Promise<unsigned long>*>>>, void>::invoke + 0x61 - [unknown source file]
> impalad ! impala::Thread::SuperviseThread + 0x228 - [unknown source file]
> impalad ! boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void (void)>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run + 0x73 - [unknown source file]
> impalad ! func@0xe4b330 + 0xd9 - [unknown source file]
> libpthread-2.17.so ! start_thread + 0xc4 - [unknown source file]
> libc-2.17.so ! __clone + 0x6c - [unknown source file]
> {code}
> {code}
> #0 0x0000000001b75ba7 in base::internal::SpinLockDelay(int volatile*, int, int) ()
> #1 0x0000000001bad949 in SpinLock::SlowLock() ()
> #2 0x0000000001bf02c8 in tcmalloc::CentralFreeList::Populate() ()
> #3 0x0000000001bf03c8 in tcmalloc::CentralFreeList::FetchFromOneSpansSafe(int, void**, void**) ()
> #4 0x0000000001bf0454 in tcmalloc::CentralFreeList::RemoveRange(void**, void**, int) ()
> #5 0x0000000001bfdbb3 in tcmalloc::ThreadCache::FetchFromCentralCache(unsigned long, unsigned long) ()
> #6 0x0000000001c0aadb in tc_malloc ()
> #7 0x0000000000a26b88 in impala::MemPool::FindChunk(long, bool) ()
> #8 0x0000000000842c35 in unsigned char* impala::MemPool::Allocate<true>(long, int) ()
> #9 0x0000000000cbc313 in impala::BaseScalarColumnReader::ReadDataPage() ()
> #10 0x0000000000cbca3f in impala::BaseScalarColumnReader::NextPage() ()
> #11 0x0000000000ccc8a8 in impala::ScalarColumnReader<int, true>::ReadNonRepeatedValueBatch(impala::MemPool*, int, int, unsigned char*, int*) ()
> #12 0x0000000000c8d254 in impala::HdfsParquetScanner::AssembleRows(std::vector<impala::ParquetColumnReader*, std::allocator<impala::ParquetColumnReader*> > const&, impala::RowBatch*, bool*) ()
> #13 0x0000000000c91e48 in impala::HdfsParquetScanner::GetNextInternal(impala::RowBatch*) ()
> #14 0x0000000000c8ca74 in impala::HdfsParquetScanner::ProcessSplit() ()
> #15 0x0000000000c5d946 in impala::HdfsScanNode::ProcessSplit(std::vector<impala::FilterContext, std::allocator<impala::FilterContext> > const&, impala::DiskIoMgr::ScanRange*) ()
> #16 0x0000000000c5f5a9 in impala::HdfsScanNode::ScannerThread() ()
> #17 0x0000000000bea1b9 in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) ()
> #18 0x0000000000beab74 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() ()
> #19 0x0000000000e4b40a in thread_proxy ()
> #20 0x00007fc0ed6c8dc5 in start_thread () from /lib64/libpthread.so.0
> #21 0x00007fc0ed3f773d in clone () from /lib64/libc.so.6
> {code}
> {code}
> Thread 1 (process 20304):
> #0 0x0000000001b75ba7 in base::internal::SpinLockDelay(int volatile*, int, int) ()
> #1 0x0000000001bad949 in SpinLock::SlowLock() ()
> #2 0x0000000001bf02c8 in tcmalloc::CentralFreeList::Populate() ()
> #3 0x0000000001bf03c8 in tcmalloc::CentralFreeList::FetchFromOneSpansSafe(int, void**, void**) ()
> #4 0x0000000001bf0454 in tcmalloc::CentralFreeList::RemoveRange(void**, void**, int) ()
> #5 0x0000000001bfdbb3 in tcmalloc::ThreadCache::FetchFromCentralCache(unsigned long, unsigned long) ()
> #6 0x0000000001c0aadb in tc_malloc ()
> #7 0x0000000000a26b88 in impala::MemPool::FindChunk(long, bool) ()
> #8 0x0000000000a39d92 in impala::RowBatch::ResizeAndAllocateTupleBuffer(impala::RuntimeState*, long*, unsigned char**) ()
> #9 0x0000000000c8d0d0 in impala::HdfsParquetScanner::AssembleRows(std::vector<impala::ParquetColumnReader*, std::allocator<impala::ParquetColumnReader*> > const&, impala::RowBatch*, bool*) ()
> #10 0x0000000000c91e48 in impala::HdfsParquetScanner::GetNextInternal(impala::RowBatch*) ()
> #11 0x0000000000c8ca74 in impala::HdfsParquetScanner::ProcessSplit() ()
> #12 0x0000000000c5d946 in impala::HdfsScanNode::ProcessSplit(std::vector<impala::FilterContext, std::allocator<impala::FilterContext> > const&, impala::DiskIoMgr::ScanRange*) ()
> #13 0x0000000000c5f5a9 in impala::HdfsScanNode::ScannerThread() ()
> #14 0x0000000000bea1b9 in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) ()
> #15 0x0000000000beab74 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() ()
> #16 0x0000000000e4b40a in thread_proxy ()
> #17 0x00007f0c99962dc5 in start_thread () from /lib64/libpthread.so.0
> #18 0x00007f0c9969173d in clone () from /lib64/libc.so.6
> {code}
> {code}
> #0 0x0000000001b75ba7 in base::internal::SpinLockDelay(int volatile*, int, int) ()
> #1 0x0000000001bad949 in SpinLock::SlowLock() ()
> #2 0x0000000001befd38 in tcmalloc::CentralFreeList::ReleaseToSpans(void*) ()
> #3 0x0000000001befd7b in tcmalloc::CentralFreeList::ReleaseListToSpans(void*) ()
> #4 0x0000000001beffec in tcmalloc::CentralFreeList::InsertRange(void*, void*, int) ()
> #5 0x0000000001bfdd54 in tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*, unsigned long, int) ()
> #6 0x0000000001bfde1c in tcmalloc::ThreadCache::ListTooLong(tcmalloc::ThreadCache::FreeList*, unsigned long) ()
> #7 0x0000000001c0aff0 in tc_free ()
> #8 0x0000000000a25cea in impala::MemPool::FreeAll() ()
> #9 0x0000000000a39821 in impala::RowBatch::Reset() ()
> #10 0x0000000000cdda97 in impala::PartitionedAggregationNode::GetRowsStreaming(impala::RuntimeState*, impala::RowBatch*) ()
> #11 0x0000000000ce34f1 in impala::PartitionedAggregationNode::GetNextInternal(impala::RuntimeState*, impala::RowBatch*, bool*) ()
> #12 0x0000000000ce35b2 in impala::PartitionedAggregationNode::GetNext(impala::RuntimeState*, impala::RowBatch*, bool*) ()
> #13 0x0000000000a738d8 in impala::FragmentInstanceState::ExecInternal() ()
> #14 0x0000000000a76dd9 in impala::FragmentInstanceState::Exec() ()
> #15 0x0000000000a29516 in impala::QueryState::ExecFInstance(impala::FragmentInstanceState*) ()
> #16 0x0000000000bea1b9 in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) ()
> #17 0x0000000000beab74 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() ()
> #18 0x0000000000e4b40a in thread_proxy ()
> #19 0x00007f5f74b03dc5 in start_thread () from /lib64/libpthread.so.0
> #20 0x00007f5f7483273d in clone () from /lib64/libc.so.6
> {code}
> {code}
> #0 0x0000000001b75ba7 in base::internal::SpinLockDelay(int volatile*, int, int) ()
> #1 0x0000000001bad949 in SpinLock::SlowLock() ()
> #2 0x0000000001bf02c8 in tcmalloc::CentralFreeList::Populate() ()
> #3 0x0000000001bf03c8 in tcmalloc::CentralFreeList::FetchFromOneSpansSafe(int, void**, void**) ()
> #4 0x0000000001bf0454 in tcmalloc::CentralFreeList::RemoveRange(void**, void**, int) ()
> #5 0x0000000001bfdbb3 in tcmalloc::ThreadCache::FetchFromCentralCache(unsigned long, unsigned long) ()
> #6 0x0000000001c0aadb in tc_malloc ()
> #7 0x0000000000a26b88 in impala::MemPool::FindChunk(long, bool) ()
> #8 0x0000000000a39d92 in impala::RowBatch::ResizeAndAllocateTupleBuffer(impala::RuntimeState*, long*, unsigned char**) ()
> #9 0x0000000000c8d0d0 in impala::HdfsParquetScanner::AssembleRows(std::vector<impala::ParquetColumnReader*, std::allocator<impala::ParquetColumnReader*> > const&, impala::RowBatch*, bool*) ()
> #10 0x0000000000c91e48 in impala::HdfsParquetScanner::GetNextInternal(impala::RowBatch*) ()
> #11 0x0000000000c8ca74 in impala::HdfsParquetScanner::ProcessSplit() ()
> #12 0x0000000000c5d946 in impala::HdfsScanNode::ProcessSplit(std::vector<impala::FilterContext, std::allocator<impala::FilterContext> > const&, impala::DiskIoMgr::ScanRange*) ()
> #13 0x0000000000c5f5a9 in impala::HdfsScanNode::ScannerThread() ()
> #14 0x0000000000bea1b9 in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) ()
> #15 0x0000000000beab74 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() ()
> #16 0x0000000000e4b40a in thread_proxy ()
> #17 0x00007f0c99962dc5 in start_thread () from /lib64/libpthread.so.0
> #18 0x00007f0c9969173d in clone () from /lib64/libc.so.6
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)