You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2024/09/12 11:02:42 UTC

[Impala-ASF-CR] IMPALA-13378: Verify tuple ids in descriptor table received in executor side

Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21794


Change subject: IMPALA-13378: Verify tuple ids in descriptor table received in executor side
......................................................................

IMPALA-13378: Verify tuple ids in descriptor table received in executor side

It's suspicious that KRPC could receive incomplete sidecar data
(KUDU-3582). TQueryCtx and TExecPlanFragmentInfo are both sent as
sidecars. Executors should check whether tuple ids in
TExecPlanFragmentInfo are consistent with the descriptor table
deserialized from TQueryCtx.

This patch adds the check when launching the fragment instance threads.

Tests
 - Ran TPCH queries locally
 - TODO: run CORE tests

Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
---
M be/src/runtime/query-state.cc
1 file changed, 17 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/21794/1
-- 
To view, visit http://gerrit.cloudera.org:8080/21794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Gerrit-Change-Number: 21794
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-13378: Verify tuple ids in descriptor table received in executor side

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21794 )

Change subject: IMPALA-13378: Verify tuple ids in descriptor table received in executor side
......................................................................

IMPALA-13378: Verify tuple ids in descriptor table received in executor side

It's suspicious that KRPC could receive incomplete sidecar data
(KUDU-3582). TQueryCtx and TExecPlanFragmentInfo are both sent as
sidecars. Executors should check whether tuple ids in
TExecPlanFragmentInfo are consistent with the descriptor table
deserialized from TQueryCtx.

This patch adds the check when launching the fragment instance threads.

Tests
 - ran CORE tests

Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Reviewed-on: http://gerrit.cloudera.org:8080/21794
Reviewed-by: Wenzhe Zhou <wz...@cloudera.com>
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Michael Smith <mi...@cloudera.com>
---
M be/src/runtime/query-state.cc
1 file changed, 17 insertions(+), 0 deletions(-)

Approvals:
  Wenzhe Zhou: Looks good to me, but someone else must approve
  Michael Smith: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/21794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Gerrit-Change-Number: 21794
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-13378: Verify tuple ids in descriptor table received in executor side

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21794 )

Change subject: IMPALA-13378: Verify tuple ids in descriptor table received in executor side
......................................................................


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10947/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/21794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Gerrit-Change-Number: 21794
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Sep 2024 23:01:25 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13378: Verify tuple ids in descriptor table received in executor side

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell, Wenzhe Zhou, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/21794

to look at the new patch set (#2).

Change subject: IMPALA-13378: Verify tuple ids in descriptor table received in executor side
......................................................................

IMPALA-13378: Verify tuple ids in descriptor table received in executor side

It's suspicious that KRPC could receive incomplete sidecar data
(KUDU-3582). TQueryCtx and TExecPlanFragmentInfo are both sent as
sidecars. Executors should check whether tuple ids in
TExecPlanFragmentInfo are consistent with the descriptor table
deserialized from TQueryCtx.

This patch adds the check when launching the fragment instance threads.

Tests
 - ran CORE tests

Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
---
M be/src/runtime/query-state.cc
1 file changed, 17 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/21794/2
-- 
To view, visit http://gerrit.cloudera.org:8080/21794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Gerrit-Change-Number: 21794
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-13378: Verify tuple ids in descriptor table received in executor side

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/21794 )

Change subject: IMPALA-13378: Verify tuple ids in descriptor table received in executor side
......................................................................


Patch Set 1: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/21794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Gerrit-Change-Number: 21794
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Sep 2024 17:25:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13378: Verify tuple ids in descriptor table received in executor side

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21794 )

Change subject: IMPALA-13378: Verify tuple ids in descriptor table received in executor side
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/16933/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Gerrit-Change-Number: 21794
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Sep 2024 11:34:16 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13378: Verify tuple ids in descriptor table received in executor side

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21794 )

Change subject: IMPALA-13378: Verify tuple ids in descriptor table received in executor side
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/16942/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Gerrit-Change-Number: 21794
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 13 Sep 2024 08:43:45 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13378: Verify tuple ids in descriptor table received in executor side

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21794 )

Change subject: IMPALA-13378: Verify tuple ids in descriptor table received in executor side
......................................................................


Patch Set 2: Verified+1 Code-Review+2

Carry verified for commit message update.


-- 
To view, visit http://gerrit.cloudera.org:8080/21794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Gerrit-Change-Number: 21794
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 13 Sep 2024 17:14:00 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13378: Verify tuple ids in descriptor table received in executor side

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/21794 )

Change subject: IMPALA-13378: Verify tuple ids in descriptor table received in executor side
......................................................................


Patch Set 2: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/21794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Gerrit-Change-Number: 21794
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 13 Sep 2024 14:50:09 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13378: Verify tuple ids in descriptor table received in executor side

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21794 )

Change subject: IMPALA-13378: Verify tuple ids in descriptor table received in executor side
......................................................................


Patch Set 1: Code-Review+1

Seems like a reasonable check.


-- 
To view, visit http://gerrit.cloudera.org:8080/21794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Gerrit-Change-Number: 21794
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Sep 2024 16:51:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13378: Verify tuple ids in descriptor table received in executor side

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21794 )

Change subject: IMPALA-13378: Verify tuple ids in descriptor table received in executor side
......................................................................


Patch Set 1: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/21794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0c489d3bff7ae08813271c65086ea6b238420e47
Gerrit-Change-Number: 21794
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 13 Sep 2024 04:11:07 +0000
Gerrit-HasComments: No