You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Anonymous Coward (Code Review)" <ge...@cloudera.org> on 2023/02/06 08:09:35 UTC
[Impala-ASF-CR] IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT DOP>1
pranav.lodha@cloudera.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19474
Change subject: IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT_DOP>1
......................................................................
IMPALA-11803: Impalad hit DCHECK when running union on
empty table with MT_DOP>1
The error was caused as the value of useMtScanNode_
was not getting set for empty tables. The fix is also
verified with an end-to-end test.
Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
---
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M tests/custom_cluster/test_mt_dop.py
2 files changed, 14 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/19474/2
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT DOP>1
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
pranav.lodha@cloudera.com has removed Joe McDonnell from this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT_DOP>1
......................................................................
Removed reviewer Joe McDonnell.
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
pranav.lodha@cloudera.com has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
The DCHECK was caused as the value of useMtScanNode_
was not getting set for empty tables. This was because of
a condition for empty tables that returned from the
computeNodeResourceProfile() function before setting up
useMtScanNode_. It got rectified by placing the
assignment of useMtScanNode_ before that condition. The
fix is also verified with an end-to-end test in test_mt_dop.py.
Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
---
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M tests/custom_cluster/test_mt_dop.py
2 files changed, 11 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/19474/4
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 7: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 7
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 28 Feb 2023 18:46:16 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT DOP>1
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
pranav.lodha@cloudera.com has removed Joe McDonnell from this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT_DOP>1
......................................................................
Removed reviewer Joe McDonnell.
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT DOP>1
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 3:
(4 comments)
Thanks for fixing this.
http://gerrit.cloudera.org:8080/#/c/19474/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19474/3//COMMIT_MSG@12
PS3, Line 12: returned the
Nit: returned from the computeNodeResourceProfile() function
http://gerrit.cloudera.org:8080/#/c/19474/3//COMMIT_MSG@12
PS3, Line 12:
Nit: unnecessary leading space here and in the following lines.
http://gerrit.cloudera.org:8080/#/c/19474/3//COMMIT_MSG@16
PS3, Line 16: fix is also verified with an end-to-end test.
Could mention that the added test is in test_mt_dop.py.
http://gerrit.cloudera.org:8080/#/c/19474/3/tests/custom_cluster/test_mt_dop.py
File tests/custom_cluster/test_mt_dop.py:
http://gerrit.cloudera.org:8080/#/c/19474/3/tests/custom_cluster/test_mt_dop.py@70
PS3, Line 70: """ When used in DEBUG build, impalad crashed while running union on
We could include the Jira number here (IMPALA-11803) as a reference. The comment could for example begin with
"Regression test for IMPALA-11803"
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 23 Feb 2023 10:49:13 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 7:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9079/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 7
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 28 Feb 2023 13:43:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 6:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12452/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 27 Feb 2023 10:46:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 6: Code-Review+1
Thanks Pranav!
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 27 Feb 2023 10:31:23 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT DOP>1
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG@7
PS2, Line 7: Impalad hit DCHECK when running union on
: empty table with MT_DOP>1
> I've updated the description, should I change the title of the jira as well
No, we just need to change the commit title. The JIRA title is ok to just describe the problem. The commit title would be nice to focus on how the issue is fixed, e.g. "Fix hitting DCHECK when running union on empty table with MT_DOP>1"
BTW, the title should be in one line.
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG@11
PS2, Line 11: w
> Done
Not done yet in other lines..
http://gerrit.cloudera.org:8080/#/c/19474/3/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:
http://gerrit.cloudera.org:8080/#/c/19474/3/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@2095
PS3, Line 2095: useMtScanNode_ = queryOptions.mt_dop > 0;
Could you add a comment like this? So we won't make the same mistake when adding codes in the future.
// update 'useMtScanNode_' before any return cases. It's used in BE.
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Wed, 15 Feb 2023 08:56:47 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT DOP>1
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
pranav.lodha@cloudera.com has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT_DOP>1
......................................................................
IMPALA-11803: Impalad hit DCHECK when running union on
empty table with MT_DOP>1
The DCHECK was caused as the value of useMtScanNode_
was not getting set for empty tables. This was because of
a condition for empty tables that returned the
computeNodeResourceProfile() function before setting up
useMtScanNode_. It got rectified by placing the
assignment of useMtScanNode_ before that condition. The
fix is also verified with an end-to-end test.
Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
---
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M tests/custom_cluster/test_mt_dop.py
2 files changed, 9 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/19474/3
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT DOP>1
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
pranav.lodha@cloudera.com has removed Joe McDonnell from this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT_DOP>1
......................................................................
Removed reviewer Joe McDonnell.
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT DOP>1
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 2:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG@7
PS2, Line 7: Impalad hit DCHECK when running union on
: empty table with MT_DOP>1
nit: we'd better summarize how the issue is fixed instead of describing the issue.
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG@11
PS2, Line 11:
nit: no space at line start
http://gerrit.cloudera.org:8080/#/c/19474/2/tests/custom_cluster/test_mt_dop.py
File tests/custom_cluster/test_mt_dop.py:
http://gerrit.cloudera.org:8080/#/c/19474/2/tests/custom_cluster/test_mt_dop.py@68
PS2, Line 68: def test_mt_dop_union_empty_table(self, unique_database):
Please verify the test failed without the fix. I think it won't fail since the following annotation is missing here:
@CustomClusterTestSuite.with_args(cluster_size=1)
http://gerrit.cloudera.org:8080/#/c/19474/2/tests/custom_cluster/test_mt_dop.py@73
PS2, Line 73: self.client.execute("create table {0}.tbl (id int) stored as parquet"
: .format(unique_database))
: self.client.execute("insert into {0}.tbl values (0), (1), (2)"
: .format(unique_database))
We can use an existing table (e.g. functional.alltypestiny) instead of creating a new one.
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 06 Feb 2023 09:04:39 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
The DCHECK was caused as the value of useMtScanNode_
was not getting set for empty tables. This was because of
a condition for empty tables that returned from the
computeNodeResourceProfile() function before setting up
useMtScanNode_. It got rectified by placing the
assignment of useMtScanNode_ before that condition.
Testing: The fix is verified with an end-to-end test in test_mt_dop.py.
Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Reviewed-on: http://gerrit.cloudera.org:8080/19474
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M tests/custom_cluster/test_mt_dop.py
2 files changed, 11 insertions(+), 1 deletion(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 8
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
pranav.lodha@cloudera.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 6:
(9 comments)
> Patch Set 5:
>
> (1 comment)
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG@7
PS2, Line 7: Fix hitting DCHECK when running union on empty table with MT_DOP>1
:
> No, we just need to change the commit title. The JIRA title is ok to just d
Sure, done
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG@11
PS2, Line 11: a
> Not done yet in other lines..
Done
http://gerrit.cloudera.org:8080/#/c/19474/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19474/3//COMMIT_MSG@12
PS3, Line 12: c
> Nit: unnecessary leading space here and in the following lines.
Done
http://gerrit.cloudera.org:8080/#/c/19474/3//COMMIT_MSG@12
PS3, Line 12: on before se
> Nit: returned from the computeNodeResourceProfile() function
Done
http://gerrit.cloudera.org:8080/#/c/19474/3//COMMIT_MSG@16
PS3, Line 16: Testing: The fix is verified with an end-to-end test in test_mt_dop.py.
> Could mention that the added test is in test_mt_dop.py.
Done
http://gerrit.cloudera.org:8080/#/c/19474/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19474/4//COMMIT_MSG@14
PS4, Line 14:
:
> We usually have a "Testing" section in the commit message, you could put th
Alright, I'll take care of it from next time. Done.
http://gerrit.cloudera.org:8080/#/c/19474/5//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19474/5//COMMIT_MSG@15
PS5, Line 15:
> I usually leave an empty line before the 'Testing' section and start the te
Done
http://gerrit.cloudera.org:8080/#/c/19474/3/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:
http://gerrit.cloudera.org:8080/#/c/19474/3/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@2095
PS3, Line 2095: // Update 'useMtScanNode_' before any return cases. It's used in BE.
> Could you add a comment like this? So we won't make the same mistake when a
Sure, done.
http://gerrit.cloudera.org:8080/#/c/19474/3/tests/custom_cluster/test_mt_dop.py
File tests/custom_cluster/test_mt_dop.py:
http://gerrit.cloudera.org:8080/#/c/19474/3/tests/custom_cluster/test_mt_dop.py@70
PS3, Line 70: """ Regression test for IMPALA-11803: When used in DEBUG build,
> We could include the Jira number here (IMPALA-11803) as a reference. The co
Done
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 27 Feb 2023 10:25:55 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
pranav.lodha@cloudera.com has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
The DCHECK was caused as the value of useMtScanNode_
was not getting set for empty tables. This was because of
a condition for empty tables that returned from the
computeNodeResourceProfile() function before setting up
useMtScanNode_. It got rectified by placing the
assignment of useMtScanNode_ before that condition.
Testing: The fix is verified with an end-to-end test in test_mt_dop.py.
Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
---
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M tests/custom_cluster/test_mt_dop.py
2 files changed, 11 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/19474/6
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12450/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 27 Feb 2023 08:07:59 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 4:
(1 comment)
Thanks Pranav, just a small remark.
http://gerrit.cloudera.org:8080/#/c/19474/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19474/4//COMMIT_MSG@14
PS4, Line 14: The
: fix is also verified with an end-to-end test in test_mt_dop.py.
We usually have a "Testing" section in the commit message, you could put this sentence into it like this:
...
Testing:
The fix is verified ...
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 27 Feb 2023 09:27:55 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT DOP>1
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12333/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Wed, 08 Feb 2023 06:19:28 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 7: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 7
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 28 Feb 2023 13:43:31 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT DOP>1
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Fix hitting DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 6: Code-Review+2
LGTM. Carrying Daniel's +1. Thanks for fixing this, Pranav!
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 28 Feb 2023 13:42:52 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT DOP>1
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
pranav.lodha@cloudera.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 3:
(4 comments)
> Patch Set 2:
>
> (4 comments)
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG@7
PS2, Line 7: Impalad hit DCHECK when running union on
: empty table with MT_DOP>1
> nit: we'd better summarize how the issue is fixed instead of describing the
I've updated the description, should I change the title of the jira as well?
http://gerrit.cloudera.org:8080/#/c/19474/2//COMMIT_MSG@11
PS2, Line 11: w
> nit: no space at line start
Done
http://gerrit.cloudera.org:8080/#/c/19474/2/tests/custom_cluster/test_mt_dop.py
File tests/custom_cluster/test_mt_dop.py:
http://gerrit.cloudera.org:8080/#/c/19474/2/tests/custom_cluster/test_mt_dop.py@68
PS2, Line 68: @CustomClusterTestSuite.with_args(cluster_size=1)
> Please verify the test failed without the fix. I think it won't fail since
Done
http://gerrit.cloudera.org:8080/#/c/19474/2/tests/custom_cluster/test_mt_dop.py@73
PS2, Line 73: self.client.execute("select count(*) from (select f2 from"
: " functional.emptytable union all select id from"
: " functional.alltypestiny) t")
:
> We can use an existing table (e.g. functional.alltypestiny) instead of crea
Done
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Wed, 08 Feb 2023 06:01:47 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT DOP>1
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19474 )
Change subject: IMPALA-11803: Impalad hit DCHECK when running union on empty table with MT_DOP>1
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12314/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idbae5e1a78211327a214b2d936743bda767ae3c4
Gerrit-Change-Number: 19474
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 06 Feb 2023 08:29:54 +0000
Gerrit-HasComments: No