You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Csaba Ringhofer (Jira)" <ji...@apache.org> on 2020/01/14 21:56:00 UTC
[jira] [Resolved] (IMPALA-9249) HdfsOrcScanner crash on resolving
file schema
[ https://issues.apache.org/jira/browse/IMPALA-9249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Csaba Ringhofer resolved IMPALA-9249.
-------------------------------------
Resolution: Fixed
> HdfsOrcScanner crash on resolving file schema
> ---------------------------------------------
>
> Key: IMPALA-9249
> URL: https://issues.apache.org/jira/browse/IMPALA-9249
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Quanlong Huang
> Assignee: Csaba Ringhofer
> Priority: Blocker
> Labels: impala-orc-experimental
>
> Encounter a crash when running test_scanners_fuzz test on orc format:
> {code:java}
> F1214 12:35:18.365988 4945 orc-metadata-utils.cc:29] c749de778e550dcb:50f6705f00000001] Check failed: root.getKind() == orc::TypeKind::STRUCT (0 vs. 12)
> *** Check failure stack trace: ***
> @ 0x4d864fc google::LogMessage::Fail()
> @ 0x4d87da1 google::LogMessage::SendToLog()
> @ 0x4d85ed6 google::LogMessage::Flush()
> @ 0x4d8949d google::LogMessageFatal::~LogMessageFatal()
> @ 0x2622c73 impala::OrcMetadataUtils::BuildSchemaPaths()
> @ 0x25e49ff impala::HdfsOrcScanner::Open()
> @ 0x2570e06 impala::HdfsScanNodeBase::CreateAndOpenScannerHelper()
> @ 0x256418f impala::HdfsScanNode::ProcessSplit()
> @ 0x25636f4 impala::HdfsScanNode::ScannerThread()
> @ 0x2562a7d _ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_18ThreadResourcePoolEENKUlvE_clEv
> @ 0x2565045 _ZN5boost6detail8function26void_function_obj_invoker0IZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS3_18ThreadResourcePoolEEUlvE_vE6invokeERNS1_15function_bufferE
> @ 0x1f1e469 boost::function0<>::operator()()
> @ 0x24aaec8 impala::Thread::SuperviseThread()
> @ 0x24b324c boost::_bi::list5<>::operator()<>()
> @ 0x24b3170 boost::_bi::bind_t<>::operator()()
> @ 0x24b3133 boost::detail::thread_data<>::run()
> @ 0x3caf6e9 thread_proxy
> @ 0x7f4b5be7a6b9 start_thread
> @ 0x7f4b5863641c clone{code}
> Code snipper:
> {code:c++}
> void OrcMetadataUtils::BuildSchemaPaths(const orc::Type& root, int num_partition_keys,
> vector<SchemaPath>* paths) {
> SchemaPath path;
> paths->push_back(path);
> DCHECK_EQ(root.getKind(), orc::TypeKind::STRUCT); <-- It could be wrong for corrupt files
> int num_columns = root.getSubtypeCount();
> for (int i = 0; i < num_columns; ++i) {
> path.push_back(i + num_partition_keys);
> BuildSchemaPath(*root.getSubtype(i), &path, paths);
> path.pop_back();
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)