You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2022/03/15 08:48:01 UTC
[Impala-ASF-CR] IMPALA-9250: catch exceptions of orc::RowReader::createRowBatch
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18321
Change subject: IMPALA-9250: catch exceptions of orc::RowReader::createRowBatch
......................................................................
IMPALA-9250: catch exceptions of orc::RowReader::createRowBatch
The ORC lib uses exceptions to report failures. We are missing
exception handling in invoking orc::RowReader::createRowBatch which
requires memory allocation and could raise exceptions when it fails.
This patch simply adds a catch clause for it.
To simplify the codes, a macro is added for catching the ORC exceptions
with given message and an additional statement.
Tests:
- TODO: run test_scanner_fuzz.py 100 times
Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
---
M be/src/exec/hdfs-orc-scanner.cc
1 file changed, 32 insertions(+), 51 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/18321/1
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18321 )
Change subject: IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
......................................................................
IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
The ORC lib uses exceptions to report failures. We are missing
exception handling in invoking orc::RowReader::createRowBatch which
requires memory allocation and could raise exceptions when it fails.
This patch simply adds a catch clause for it.
To simplify the codes, a macro is added for catching the ORC exceptions
with given message format.
Tests:
- run test_scanner_fuzz.py 20 times
Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Reviewed-on: http://gerrit.cloudera.org:8080/18321
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exec/hdfs-orc-scanner.cc
1 file changed, 28 insertions(+), 51 deletions(-)
Approvals:
Csaba Ringhofer: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18321 )
Change subject: IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
......................................................................
Patch Set 5:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7941/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 18 Mar 2022 09:26:29 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18321 )
Change subject: IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
......................................................................
Patch Set 5:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/18321/3/be/src/exec/hdfs-orc-scanner.cc
File be/src/exec/hdfs-orc-scanner.cc:
http://gerrit.cloudera.org:8080/#/c/18321/3/be/src/exec/hdfs-orc-scanner.cc@51
PS3, Line 51: rn parse_status_;
> My only concern with this macro is that the name does not inform about "ret
Done
http://gerrit.cloudera.org:8080/#/c/18321/3/be/src/exec/hdfs-orc-scanner.cc@960
PS3, Line 960:
> I wonder if this is necessary - does the value of eos_ matter once an error
I checked the code paths of using eos_ and it's not used when status is non-ok. I think we can remove this.
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 18 Mar 2022 04:52:27 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18321 )
Change subject: IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 18 Mar 2022 08:11:31 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/18321 )
Change subject: IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
......................................................................
Patch Set 3: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 16 Mar 2022 15:34:17 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9250: catch exceptions of orc::RowReader::createRowBatch
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18321 )
Change subject: IMPALA-9250: catch exceptions of orc::RowReader::createRowBatch
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/10290/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 15 Mar 2022 09:08:29 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18321
to look at the new patch set (#3).
Change subject: IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
......................................................................
IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
The ORC lib uses exceptions to report failures. We are missing
exception handling in invoking orc::RowReader::createRowBatch which
requires memory allocation and could raise exceptions when it fails.
This patch simply adds a catch clause for it.
To simplify the codes, a macro is added for catching the ORC exceptions
with given message and an additional statement.
Tests:
- run test_scanner_fuzz.py 10 times
Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
---
M be/src/exec/hdfs-orc-scanner.cc
1 file changed, 32 insertions(+), 51 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/18321/3
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18321
to look at the new patch set (#5).
Change subject: IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
......................................................................
IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
The ORC lib uses exceptions to report failures. We are missing
exception handling in invoking orc::RowReader::createRowBatch which
requires memory allocation and could raise exceptions when it fails.
This patch simply adds a catch clause for it.
To simplify the codes, a macro is added for catching the ORC exceptions
with given message format.
Tests:
- run test_scanner_fuzz.py 20 times
Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
---
M be/src/exec/hdfs-orc-scanner.cc
1 file changed, 28 insertions(+), 51 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/18321/5
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18321 )
Change subject: IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
......................................................................
Patch Set 3: Code-Review+1
(2 comments)
http://gerrit.cloudera.org:8080/#/c/18321/3/be/src/exec/hdfs-orc-scanner.cc
File be/src/exec/hdfs-orc-scanner.cc:
http://gerrit.cloudera.org:8080/#/c/18321/3/be/src/exec/hdfs-orc-scanner.cc@51
PS3, Line 51: CATCH_ORC_EXCEPTIONS_WITH_STMT
My only concern with this macro is that the name does not inform about "return"ing in case of error. For example RETURN_ON_ORC_EXCEPTION_WITH_STMT would probably make the control flow clearer.
http://gerrit.cloudera.org:8080/#/c/18321/3/be/src/exec/hdfs-orc-scanner.cc@960
PS3, Line 960: eos_ = true
I wonder if this is necessary - does the value of eos_ matter once an error is returned? I also don't get why we set it for std::exception but not for ResourceError
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Mar 2022 12:27:20 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18321 )
Change subject: IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/10306/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 18 Mar 2022 05:11:59 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18321 )
Change subject: IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
......................................................................
Patch Set 5: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 18 Mar 2022 14:17:53 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18321
to look at the new patch set (#2).
Change subject: IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
......................................................................
IMPALA-11182: catch exceptions of orc::RowReader::createRowBatch
The ORC lib uses exceptions to report failures. We are missing
exception handling in invoking orc::RowReader::createRowBatch which
requires memory allocation and could raise exceptions when it fails.
This patch simply adds a catch clause for it.
To simplify the codes, a macro is added for catching the ORC exceptions
with given message and an additional statement.
Tests:
- TODO: run test_scanner_fuzz.py 100 times
Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
---
M be/src/exec/hdfs-orc-scanner.cc
1 file changed, 32 insertions(+), 51 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/18321/2
--
To view, visit http://gerrit.cloudera.org:8080/18321
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I76e36a238220e7bed1cbbdcb3fc7d35394bfa023
Gerrit-Change-Number: 18321
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>