You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Daniel Becker (JIRA)" <ji...@apache.org> on 2019/08/07 11:06:00 UTC
[jira] [Commented] (IMPALA-8840) Check failed: num_bytes <=
sizeof(T) (5 vs. 4)
[ https://issues.apache.org/jira/browse/IMPALA-8840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16901980#comment-16901980 ]
Daniel Becker commented on IMPALA-8840:
---------------------------------------
I think the fix to IMPALA-8833 solves this, but not necessarily other uses of RleBatchDecoder.
The problem is that RleBatchedReader<T>::Reset() doesn't check whether the bit width is at most sizeof(T) * 8. It DCHECKs if it is more than BatchedBitReader::MAX_BITWIDTH but that is not enough as later it unpacks values to T, which DCHECKs if the bit widths is too high for T. Also, Reset() returns nothing so we cannot signal an error.
As this interface is called by other Impala functions and not with user-provided data, I'll not add a runtime check but a DCHECK and document that the bit width should be at most sizeof(T) * 8.
> Check failed: num_bytes <= sizeof(T) (5 vs. 4)
> -----------------------------------------------
>
> Key: IMPALA-8840
> URL: https://issues.apache.org/jira/browse/IMPALA-8840
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.3.0
> Reporter: Xiaomeng Zhang
> Assignee: Daniel Becker
> Priority: Blocker
> Labels: broken-build, crash
>
> Not sure if this is due to same issue as https://issues.apache.org/jira/browse/IMPALA-8833#, the error message is a little different.
> {code:java}
> F0805 18:48:08.737411 5488 bit-stream-utils.inline.h:173] 284731e5d1aad693:05c8830200000001] Check failed: num_bytes <= sizeof(T) (8 vs. 4)
> *** Check failure stack trace: ***
> @ 0x52fb9bc google::LogMessage::Fail()
> @ 0x52fd261 google::LogMessage::SendToLog()
> @ 0x52fb396 google::LogMessage::Flush()
> @ 0x52fe95d google::LogMessageFatal::~LogMessageFatal()
> @ 0x2b2b867 impala::BatchedBitReader::GetBytes<>()
> @ 0x2aeda65 impala::RleBatchDecoder<>::NextCounts()
> @ 0x2a82896 impala::RleBatchDecoder<>::NextNumRepeats()
> @ 0x2b1927f impala::ScalarColumnReader<>::ReadSlotsNoConversion()
> @ 0x2ac7c2c impala::ScalarColumnReader<>::ReadSlots()
> @ 0x2a7b861 impala::ScalarColumnReader<>::MaterializeValueBatchRepeatedDefLevel()
> @ 0x2a5b3b0 impala::ScalarColumnReader<>::ReadValueBatch<>()
> @ 0x2a256a4 impala::ScalarColumnReader<>::ReadNonRepeatedValueBatch()
> @ 0x29b6eb6 impala::HdfsParquetScanner::AssembleRows()
> @ 0x29b1cf8 impala::HdfsParquetScanner::GetNextInternal()
> @ 0x29afc70 impala::HdfsParquetScanner::ProcessSplit()
> @ 0x2494bc3 impala::HdfsScanNode::ProcessSplit()
> @ 0x2493d98 impala::HdfsScanNode::ScannerThread()
> @ 0x2493121 _ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_18ThreadResourcePoolEENKUlvE_clEv
> @ 0x24956e9 _ZN5boost6detail8function26void_function_obj_invoker0IZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS3_18ThreadResourcePoolEEUlvE_vE6invokeERNS1_15function_bufferE
> @ 0x1ea0241 boost::function0<>::operator()()
> @ 0x23de77a impala::Thread::SuperviseThread()
> @ 0x23e6afe boost::_bi::list5<>::operator()<>()
> @ 0x23e6a22 boost::_bi::bind_t<>::operator()()
> @ 0x23e69e5 boost::detail::thread_data<>::run()
> @ 0x4224819 thread_proxy
> @ 0x7fc1818c5e24 start_thread
> @ 0x7fc17e01f34c __clone
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org