You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2018/05/07 22:02:00 UTC
[jira] [Resolved] (IMPALA-6946) Hit DCHECK in
impala::RleBatchDecoder::GetRepeatedValue
[ https://issues.apache.org/jira/browse/IMPALA-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-6946.
-----------------------------------
Resolution: Fixed
Fix Version/s: Impala 3.1.0
Impala 2.13.0
> Hit DCHECK in impala::RleBatchDecoder<unsigned int>::GetRepeatedValue
> ---------------------------------------------------------------------
>
> Key: IMPALA-6946
> URL: https://issues.apache.org/jira/browse/IMPALA-6946
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.11.0, Impala 3.0, Impala 2.12.0, Impala 2.13.0, Impala 3.1.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Critical
> Labels: crash
> Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> The bug comes from conversion between signed and unsigned.
> {noformat}
> DCHECK_GT(num_repeats_to_consume, 0);
> (gdb) p num_repeats_to_consume
> $2 = -1003251240
> {noformat}
> {noformat}
> #2 0x0000000001f422f0 in impala::ScalarColumnReader<impala::StringValue, (parquet::Type::type)6, true>::ReadNonRepeatedValueBatch (this=0x6f547d50, pool=0x7fb37bdbb330, max_values=0,
> tuple_size=90, tuple_mem=0x0, num_values=0x7fb37bdbb330) at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:252
> #3 0x0000000003fceea7 in google::LogMessage::Flush() ()
> #4 0x0000000003fd246e in google::LogMessageFatal::~LogMessageFatal() ()
> #5 0x0000000001f97197 in impala::RleBatchDecoder<unsigned int>::GetRepeatedValue (this=0x1803fc6a0, num_repeats_to_consume=-1003251240)
> at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/rle-encoding.h:493
> #6 0x0000000001f96d7b in impala::DictDecoder<impala::StringValue>::DecodeNextValue (this=0x1803fc698, value=0x16b7e9000)
> at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/dict-encoding.h:455
> #7 0x0000000001f94528 in impala::DictDecoder<impala::StringValue>::GetNextValue (value=0x16b7e9000, this=0x1803fc698)
> at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/dict-encoding.h:446
> #8 impala::ScalarColumnReader<impala::StringValue, (parquet::Type::type)6, true>::ReadSlot<true, false> (pool=0x6f547d98, tuple=0x16b7e9000, this=0x1803fc400)
> at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:499
> #9 impala::ScalarColumnReader<impala::StringValue, (parquet::Type::type)6, true>::MaterializeValueBatch<false, true, false> (this=0x1803fc400, pool=0x6f547d98, max_values=1, tuple_size=90,
> tuple_mem=0x16b7e9000 "", num_values=0x7fb37bdbb5f0) at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:413
> #10 0x0000000001f67b4a in impala::ScalarColumnReader<impala::StringValue, (parquet::Type::type)6, true>::MaterializeValueBatch<false, true> (this=0x1803fc400, pool=0x6f547d98, max_values=1,
> tuple_size=90, tuple_mem=0x16b7e9000 "", num_values=0x7fb37bdbb5f0) at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:436
> #11 0x0000000001f657c2 in impala::ScalarColumnReader<impala::StringValue, (parquet::Type::type)6, true>::ReadValueBatch<false> (this=0x1803fc400, pool=0x6f547d98, max_values=1, tuple_size=90,
> tuple_mem=0x16b7e9000 "", num_values=0x6f547d50) at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:354
> #12 0x0000000001f422f0 in impala::ScalarColumnReader<impala::StringValue, (parquet::Type::type)6, true>::ReadNonRepeatedValueBatch (this=0x1803fc400, pool=0x6f547d98, max_values=1,
> tuple_size=90, tuple_mem=0x16b7e9000 "", num_values=0x6f547d50) at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:252
> #13 0x0000000001d70a1c in impala::HdfsParquetScanner::AssembleRows (this=0x1a926000, column_readers=..., row_batch=0xdfc0780, skip_row_group=0x1a9261b8)
> at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-parquet-scanner.cc:936
> #14 0x0000000001d6d4f7 in impala::HdfsParquetScanner::GetNextInternal (this=0x1a926000, row_batch=0xdfc0780)
> at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-parquet-scanner.cc:434
> #15 0x0000000001d6b6c6 in impala::HdfsParquetScanner::ProcessSplit (this=0x1a926000) at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-parquet-scanner.cc:332
> #16 0x0000000001cf30e4 in impala::HdfsScanNode::ProcessSplit (this=0x10c21000, filter_ctxs=..., expr_results_pool=0x7fb37bdbc480, scan_range=0xf3276c0, scanner_thread_reservation=0x7fb37bdbc400)
> at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:482
> #17 0x0000000001cf2476 in impala::HdfsScanNode::ScannerThread (this=0x10c21000, scanner_thread_reservation=90112)
> at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:385
> #18 0x0000000001cf18a8 in impala::HdfsScanNode::<lambda()>::operator()(void) const (__closure=0x7fb37bdbcbc8)
> at /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:300
> {noformat}
> I don't believe this will cause a crash on a release build and can only be triggered by a very specific corrupt parquet file.
> It was introduced by IMPALA-4177
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)