You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Zoltán Borók-Nagy (JIRA)" <ji...@apache.org> on 2017/11/02 16:18:00 UTC
[jira] [Closed] (IMPALA-1291) Parquet read fails if io buffer size
is less than the footer size
[ https://issues.apache.org/jira/browse/IMPALA-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zoltán Borók-Nagy closed IMPALA-1291.
-------------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.11.0
> Parquet read fails if io buffer size is less than the footer size
> -----------------------------------------------------------------
>
> Key: IMPALA-1291
> URL: https://issues.apache.org/jira/browse/IMPALA-1291
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 1.4.1
> Reporter: Nong Li
> Assignee: Zoltán Borók-Nagy
> Priority: Minor
> Labels: crash, downgraded
> Fix For: Impala 2.11.0
>
>
> This will practically only happen if --read_size is manually set to be very low (e.g. for testing) so that is is less than HdfsParquetScanner::FOOTER_SIZE (which is currently 102400). It fails with this dcheck:
> {code}
> F0922 12:20:03.093201 24325 hdfs-parquet-scanner.cc:897] Check failed: stream_->eosr()
> *** Check failure stack trace: ***
> @ 0xf251dd google::LogMessage::Fail()
> @ 0xf2797f google::LogMessage::SendToLog()
> @ 0xf24dcb google::LogMessage::Flush()
> @ 0xf2820d google::LogMessageFatal::~LogMessageFatal()
> @ 0x7f68a73e9f25 impala::HdfsParquetScanner::ProcessFooter()
> @ 0x7f68a73e8af6 impala::HdfsParquetScanner::ProcessSplit()
> @ 0x7f68a7390e8d impala::HdfsScanNode::ScannerThread()
> @ 0x7f68a73a8dae boost::_mfi::mf0<>::operator()()
> @ 0x7f68a73a8436 boost::_bi::list1<>::operator()<>()
> @ 0x7f68a73a7093 boost::_bi::bind_t<>::operator()()
> @ 0x7f68a73a594b boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x7f68a5be9d6c boost::function0<>::operator()()
> @ 0x7f68a47cd33c impala::Thread::SuperviseThread()
> @ 0x7f68a47d5d6a boost::_bi::list4<>::operator()<>()
> @ 0x7f68a47d5cb3 boost::_bi::bind_t<>::operator()()
> @ 0x7f68a47d5c46 boost::detail::thread_data<>::run()
> @ 0x7f68a3cf8ce9 (unknown)
> @ 0x7f68a3ad6e9a start_thread
> @ 0x7f68a116accd (unknown)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)