You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Jim Apple (Code Review)" <ge...@cloudera.org> on 2019/07/05 00:27:53 UTC
[Impala-ASF-CR] IMPALA-5031: out-of-range enum values are undefined behavior
Jim Apple has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13805
Change subject: IMPALA-5031: out-of-range enum values are undefined behavior
......................................................................
IMPALA-5031: out-of-range enum values are undefined behavior
This patch fixes an out-of-range enum value in the end-to-end tests.
The [expr] section of the C++14 standard indicates that out-of-range
enum values are undefined behavior: "If during the evaluation of an
expression, the result is not mathematically defined or not in the
range of representable values for its type, the behavior is
undefined."
The [decl.enum] section explains what values are "in the range of
representable values for its type": "[F]or an enumeration where emin
is the smallest enumerator and emax is the largest, the values of the
enumeration are the values in the range bmin to bmax, defined as
follows: Let K be 1 for a two's complement representation and 0 for a
one's complement or sign-magnitude representation. bmax is the
smallest value greater than or equal to max(|emin| - K, |emax|) and
equal to 2M-1, where M is a non-negative integer. bmin is zero if emin
is non-negative and -(bmax+K) otherwise."
The Parquet PageType enum has emin = 0 and emax = 3, so bmin = 0 and
bmax = 3. The out-of-range value in the tests is 4294967249, and is
therefore undefined behavior. The interesting part of the backtrace
is:
parquet/parquet-column-readers.cc:1269:24: runtime error: load of
value 4294967249, which is not a valid value for type
'PageType::type'
#0 BaseScalarColumnReader::InitDictionary()
parquet/parquet-column-readers.cc:1269:24
#1 BaseScalarColumnReader::InitDictionaries(
vector<BaseScalarColumnReader*>)
parquet/parquet-column-readers.cc:1381:53
#2 HdfsParquetScanner::NextRowGroup()
parquet/hdfs-parquet-scanner.cc:678:14
#3 HdfsParquetScanner::GetNextInternal(RowBatch*)
parquet/hdfs-parquet-scanner.cc:437:45
#4 HdfsParquetScanner::ProcessSplit()
parquet/hdfs-parquet-scanner.cc:353:21
#5 HdfsScanNode::ProcessSplit(vector<FilterContext> const&,
MemPool*, io::ScanRange*, long*) exec/hdfs-scan-node.cc:514:21
#6 HdfsScanNode::ScannerThread(bool, long)
hdfs-scan-node.cc:415:7
#7 HdfsScanNode::ThreadTokenAvailableCb(ThreadResourcePool*)
::$_0::operator()() const hdfs-scan-node.cc:337:13
Change-Id: I2d126a8f3e5910d23088a3f916c4cf31aac28d95
---
M be/src/exec/parquet/parquet-column-readers.cc
1 file changed, 4 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/13805/1
--
To view, visit http://gerrit.cloudera.org:8080/13805
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2d126a8f3e5910d23088a3f916c4cf31aac28d95
Gerrit-Change-Number: 13805
Gerrit-PatchSet: 1
Gerrit-Owner: Jim Apple <jb...@apache.org>
[Impala-ASF-CR] IMPALA-5031: out-of-range enum values are undefined behavior
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13805 )
Change subject: IMPALA-5031: out-of-range enum values are undefined behavior
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/3828/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/13805
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2d126a8f3e5910d23088a3f916c4cf31aac28d95
Gerrit-Change-Number: 13805
Gerrit-PatchSet: 1
Gerrit-Owner: Jim Apple <jb...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 Jul 2019 01:07:23 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5031: out-of-range enum values are undefined behavior
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13805 )
Change subject: IMPALA-5031: out-of-range enum values are undefined behavior
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/13805
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2d126a8f3e5910d23088a3f916c4cf31aac28d95
Gerrit-Change-Number: 13805
Gerrit-PatchSet: 2
Gerrit-Owner: Jim Apple <jb...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sun, 21 Jul 2019 00:28:36 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5031: out-of-range enum values are undefined behavior
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13805 )
Change subject: IMPALA-5031: out-of-range enum values are undefined behavior
......................................................................
Patch Set 2: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/13805
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2d126a8f3e5910d23088a3f916c4cf31aac28d95
Gerrit-Change-Number: 13805
Gerrit-PatchSet: 2
Gerrit-Owner: Jim Apple <jb...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sun, 21 Jul 2019 06:38:45 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5031: out-of-range enum values are undefined behavior
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13805 )
Change subject: IMPALA-5031: out-of-range enum values are undefined behavior
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4629/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/13805
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2d126a8f3e5910d23088a3f916c4cf31aac28d95
Gerrit-Change-Number: 13805
Gerrit-PatchSet: 2
Gerrit-Owner: Jim Apple <jb...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sun, 21 Jul 2019 00:28:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5031: out-of-range enum values are undefined behavior
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13805 )
Change subject: IMPALA-5031: out-of-range enum values are undefined behavior
......................................................................
IMPALA-5031: out-of-range enum values are undefined behavior
This patch fixes an out-of-range enum value in the end-to-end tests.
The [expr] section of the C++14 standard indicates that out-of-range
enum values are undefined behavior: "If during the evaluation of an
expression, the result is not mathematically defined or not in the
range of representable values for its type, the behavior is
undefined."
The [decl.enum] section explains what values are "in the range of
representable values for its type": "[F]or an enumeration where emin
is the smallest enumerator and emax is the largest, the values of the
enumeration are the values in the range bmin to bmax, defined as
follows: Let K be 1 for a two's complement representation and 0 for a
one's complement or sign-magnitude representation. bmax is the
smallest value greater than or equal to max(|emin| - K, |emax|) and
equal to 2M-1, where M is a non-negative integer. bmin is zero if emin
is non-negative and -(bmax+K) otherwise."
The Parquet PageType enum has emin = 0 and emax = 3, so bmin = 0 and
bmax = 3. The out-of-range value in the tests is 4294967249, and is
therefore undefined behavior. The interesting part of the backtrace
is:
parquet/parquet-column-readers.cc:1269:24: runtime error: load of
value 4294967249, which is not a valid value for type
'PageType::type'
#0 BaseScalarColumnReader::InitDictionary()
parquet/parquet-column-readers.cc:1269:24
#1 BaseScalarColumnReader::InitDictionaries(
vector<BaseScalarColumnReader*>)
parquet/parquet-column-readers.cc:1381:53
#2 HdfsParquetScanner::NextRowGroup()
parquet/hdfs-parquet-scanner.cc:678:14
#3 HdfsParquetScanner::GetNextInternal(RowBatch*)
parquet/hdfs-parquet-scanner.cc:437:45
#4 HdfsParquetScanner::ProcessSplit()
parquet/hdfs-parquet-scanner.cc:353:21
#5 HdfsScanNode::ProcessSplit(vector<FilterContext> const&,
MemPool*, io::ScanRange*, long*) exec/hdfs-scan-node.cc:514:21
#6 HdfsScanNode::ScannerThread(bool, long)
hdfs-scan-node.cc:415:7
#7 HdfsScanNode::ThreadTokenAvailableCb(ThreadResourcePool*)
::$_0::operator()() const hdfs-scan-node.cc:337:13
Change-Id: I2d126a8f3e5910d23088a3f916c4cf31aac28d95
Reviewed-on: http://gerrit.cloudera.org:8080/13805
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exec/parquet/parquet-column-readers.cc
1 file changed, 4 insertions(+), 1 deletion(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/13805
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I2d126a8f3e5910d23088a3f916c4cf31aac28d95
Gerrit-Change-Number: 13805
Gerrit-PatchSet: 3
Gerrit-Owner: Jim Apple <jb...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-5031: out-of-range enum values are undefined behavior
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13805 )
Change subject: IMPALA-5031: out-of-range enum values are undefined behavior
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/13805
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2d126a8f3e5910d23088a3f916c4cf31aac28d95
Gerrit-Change-Number: 13805
Gerrit-PatchSet: 1
Gerrit-Owner: Jim Apple <jb...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sun, 21 Jul 2019 00:22:50 +0000
Gerrit-HasComments: No