You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2019/12/14 13:21:00 UTC
[jira] [Created] (IMPALA-9249) HdfsOrcScanner crash on resolving
file schema
Quanlong Huang created IMPALA-9249:
--------------------------------------
Summary: HdfsOrcScanner crash on resolving file schema
Key: IMPALA-9249
URL: https://issues.apache.org/jira/browse/IMPALA-9249
Project: IMPALA
Issue Type: Bug
Components: Backend
Reporter: Quanlong Huang
Encounter a crash when running test_scanners_fuzz test on orc format:
{code:java}
F1214 12:35:18.365988 4945 orc-metadata-utils.cc:29] c749de778e550dcb:50f6705f00000001] Check failed: root.getKind() == orc::TypeKind::STRUCT (0 vs. 12)
*** Check failure stack trace: ***
@ 0x4d864fc google::LogMessage::Fail()
@ 0x4d87da1 google::LogMessage::SendToLog()
@ 0x4d85ed6 google::LogMessage::Flush()
@ 0x4d8949d google::LogMessageFatal::~LogMessageFatal()
@ 0x2622c73 impala::OrcMetadataUtils::BuildSchemaPaths()
@ 0x25e49ff impala::HdfsOrcScanner::Open()
@ 0x2570e06 impala::HdfsScanNodeBase::CreateAndOpenScannerHelper()
@ 0x256418f impala::HdfsScanNode::ProcessSplit()
@ 0x25636f4 impala::HdfsScanNode::ScannerThread()
@ 0x2562a7d _ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_18ThreadResourcePoolEENKUlvE_clEv
@ 0x2565045 _ZN5boost6detail8function26void_function_obj_invoker0IZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS3_18ThreadResourcePoolEEUlvE_vE6invokeERNS1_15function_bufferE
@ 0x1f1e469 boost::function0<>::operator()()
@ 0x24aaec8 impala::Thread::SuperviseThread()
@ 0x24b324c boost::_bi::list5<>::operator()<>()
@ 0x24b3170 boost::_bi::bind_t<>::operator()()
@ 0x24b3133 boost::detail::thread_data<>::run()
@ 0x3caf6e9 thread_proxy
@ 0x7f4b5be7a6b9 start_thread
@ 0x7f4b5863641c clone{code}
Code snipper:
{code:c++}
void OrcMetadataUtils::BuildSchemaPaths(const orc::Type& root, int num_partition_keys,
vector<SchemaPath>* paths) {
SchemaPath path;
paths->push_back(path);
DCHECK_EQ(root.getKind(), orc::TypeKind::STRUCT); <-- It could be wrong for corrupt files
int num_columns = root.getSubtypeCount();
for (int i = 0; i < num_columns; ++i) {
path.push_back(i + num_partition_keys);
BuildSchemaPath(*root.getSubtype(i), &path, paths);
path.pop_back();
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)