You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2020/09/22 13:00:41 UTC

[Impala-ASF-CR] IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16489


Change subject: IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling
......................................................................

IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

BufferedPlanRootSink has a Promise, all_results_spooled_, that could be
accessed by different threads, e.g. the fragment execution thread and
cancellation threads. The main purpose of setting this Promise is to
unblock the coordinator if it's waiting for this. It doesn't affect
correctness if an existing value is overwritten. Only logging is affected.
So we can simply declare this Promise's mode to be MULTIPLE_PRODUCER to
avoid hitting the DCHECK in Promise.Set().

Tests:
 - Run TestResultSpoolingFailpoints::test_failpoints for more than 4000
   iterations

Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
---
M be/src/exec/buffered-plan-root-sink.cc
M be/src/exec/buffered-plan-root-sink.h
2 files changed, 9 insertions(+), 3 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/16489/1
-- 
To view, visit http://gerrit.cloudera.org:8080/16489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Gerrit-Change-Number: 16489
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16489 )

Change subject: IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling
......................................................................


Patch Set 2: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/16489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Gerrit-Change-Number: 16489
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 24 Sep 2020 05:04:16 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16489 )

Change subject: IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6470/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/16489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Gerrit-Change-Number: 16489
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 23 Sep 2020 23:48:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16489 )

Change subject: IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/7247/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/16489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Gerrit-Change-Number: 16489
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 23 Sep 2020 02:40:40 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16489 )

Change subject: IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling
......................................................................

IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

BufferedPlanRootSink has a Promise, all_results_spooled_, that could be
accessed by different threads, e.g. the fragment execution thread and
cancellation threads. The main purpose of setting this Promise is to
unblock the coordinator if it's waiting for this. So we can simply
declare this Promise's mode to be MULTIPLE_PRODUCER to avoid hitting the
DCHECK in Promise.Set().

Tests:
 - Run TestResultSpoolingFailpoints::test_failpoints for more than 4000
   iterations

Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Reviewed-on: http://gerrit.cloudera.org:8080/16489
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exec/buffered-plan-root-sink.cc
M be/src/exec/buffered-plan-root-sink.h
2 files changed, 11 insertions(+), 15 deletions(-)

Approvals:
  Tim Armstrong: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/16489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Gerrit-Change-Number: 16489
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/16489 )

Change subject: IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/16489/1/be/src/exec/buffered-plan-root-sink.cc
File be/src/exec/buffered-plan-root-sink.cc:

http://gerrit.cloudera.org:8080/#/c/16489/1/be/src/exec/buffered-plan-root-sink.cc@168
PS1, Line 168:     MonotonicStopWatch wait_timeout_timer;
> I think we can just remove the IsSet() check here and elsewhere. The old va
Oh yes! Thanks for pointing out this! Also removed the new comment which reveals my wrong understanding on Promise. No overwritting will happen...



-- 
To view, visit http://gerrit.cloudera.org:8080/16489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Gerrit-Change-Number: 16489
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 23 Sep 2020 02:19:08 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16489 )

Change subject: IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling
......................................................................


Patch Set 2: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/16489/1/be/src/exec/buffered-plan-root-sink.cc
File be/src/exec/buffered-plan-root-sink.cc:

http://gerrit.cloudera.org:8080/#/c/16489/1/be/src/exec/buffered-plan-root-sink.cc@168
PS1, Line 168:     MonotonicStopWatch wait_timeout_timer;
> Oh yes! Thanks for pointing out this! Also removed the new comment which re
Back in my PhD days I spent a bunch of time doing a literature survey on promises and related data types so this is kinda one of my pet topics :).



-- 
To view, visit http://gerrit.cloudera.org:8080/16489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Gerrit-Change-Number: 16489
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 23 Sep 2020 05:54:14 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/16489 )

Change subject: IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/16489/1/be/src/exec/buffered-plan-root-sink.cc
File be/src/exec/buffered-plan-root-sink.cc:

http://gerrit.cloudera.org:8080/#/c/16489/1/be/src/exec/buffered-plan-root-sink.cc@168
PS1, Line 168:     MonotonicStopWatch wait_timeout_timer;
> Back in my PhD days I spent a bunch of time doing a literature survey on pr
Haha, awesome! :D



-- 
To view, visit http://gerrit.cloudera.org:8080/16489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Gerrit-Change-Number: 16489
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 23 Sep 2020 23:47:53 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16489

to look at the new patch set (#2).

Change subject: IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling
......................................................................

IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

BufferedPlanRootSink has a Promise, all_results_spooled_, that could be
accessed by different threads, e.g. the fragment execution thread and
cancellation threads. The main purpose of setting this Promise is to
unblock the coordinator if it's waiting for this. So we can simply
declare this Promise's mode to be MULTIPLE_PRODUCER to avoid hitting the
DCHECK in Promise.Set().

Tests:
 - Run TestResultSpoolingFailpoints::test_failpoints for more than 4000
   iterations

Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
---
M be/src/exec/buffered-plan-root-sink.cc
M be/src/exec/buffered-plan-root-sink.h
2 files changed, 11 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/16489/2
-- 
To view, visit http://gerrit.cloudera.org:8080/16489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Gerrit-Change-Number: 16489
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16489 )

Change subject: IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/16489/1/be/src/exec/buffered-plan-root-sink.cc
File be/src/exec/buffered-plan-root-sink.cc:

http://gerrit.cloudera.org:8080/#/c/16489/1/be/src/exec/buffered-plan-root-sink.cc@168
PS1, Line 168:   if (!all_results_spooled_.IsSet()) {
I think we can just remove the IsSet() check here and elsewhere. The old value is not overwritten by Set() so this is redundant.



-- 
To view, visit http://gerrit.cloudera.org:8080/16489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Gerrit-Change-Number: 16489
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 22 Sep 2020 18:09:47 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16489 )

Change subject: IMPALA-10183: Fix hitting DCHECK when cancelling a query with result spooling
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/7236/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/16489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaba0ed729ef984f9c51347df02e9fb6f90bc71e0
Gerrit-Change-Number: 16489
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 22 Sep 2020 13:21:06 +0000
Gerrit-HasComments: No