You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Xianqing He (Code Review)" <ge...@cloudera.org> on 2021/07/27 10:06:51 UTC

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query 

Xianqing He has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17729


Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................

IMPALA-10825: fix impalad crashes when closing the retrying query

The crash happens when canceling the retrying query in web UI.
The canceling action will call ImpalaServer#UnregisterQuery.
The QueryDriver will be null if the query has already been unregistered.

Testing:
 Add test in tests/custom_cluster/test_query_retries.py and manually
tested 100 times to make sure that there was no Impalad crash

Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
---
M be/src/runtime/query-driver.cc
M tests/custom_cluster/test_query_retries.py
2 files changed, 37 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/17729/1
-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 1
Gerrit-Owner: Xianqing He <he...@126.com>

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 7:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7382/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 7
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Wed, 11 Aug 2021 00:22:07 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 6: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7380/


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 6
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Tue, 10 Aug 2021 09:40:08 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 8:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7390/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 8
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Thu, 12 Aug 2021 01:38:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 5: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7373/


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 5
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Thu, 05 Aug 2021 13:17:04 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 8: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 8
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Thu, 12 Aug 2021 07:38:38 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Qifan Chen (Code Review)" <ge...@cloudera.org>.
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 5:

The failed verification build can be found here: https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/4548/#showFailuresLink

Looks like impalad crashed.


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 5
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Thu, 05 Aug 2021 20:40:55 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9182/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 1
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 27 Jul 2021 10:28:55 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 4: Code-Review+2

Thanks for fixing this! Carry on Qifan's +1 to +2.


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 4
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Thu, 05 Aug 2021 07:03:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 6:

> Patch Set 6: Verified-1
> 
> Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7380/

The failure seems unrelated to this patch: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/14539/testReport/junit/metadata.test_hms_integration/TestHmsIntegrationSanity/test_sanity_protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/

BTW, the failure of PS5 may already be fixed by IMPALA-10808. So rebase helps.


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 6
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Wed, 11 Aug 2021 00:21:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Xianqing He (Code Review)" <ge...@cloudera.org>.
Xianqing He has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................

IMPALA-10825: fix impalad crashes when closing the retrying query

The crash happens when canceling the retrying query in web UI.
The canceling action will call ImpalaServer#UnregisterQuery.
The QueryDriver will be null if the query has already been unregistered.

Testing:
 Add test in tests/custom_cluster/test_query_retries.py and manually
tested 100 times to make sure that there was no Impalad crash

Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
---
M be/src/runtime/query-driver.cc
M tests/custom_cluster/test_query_retries.py
2 files changed, 36 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/17729/3
-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 3
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Xianqing He (Code Review)" <ge...@cloudera.org>.
Xianqing He has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 3:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/17729/2/be/src/runtime/query-driver.cc
File be/src/runtime/query-driver.cc:

http://gerrit.cloudera.org:8080/#/c/17729/2/be/src/runtime/query-driver.cc@190
PS2, Line 190:     // Update the state and then schedule the retry asynchronously.
> Should the MarkAsRetrying() call above be skipped or reset in this case?
Yeah, I agree. Done.


http://gerrit.cloudera.org:8080/#/c/17729/2/tests/custom_cluster/test_query_retries.py
File tests/custom_cluster/test_query_retries.py:

http://gerrit.cloudera.org:8080/#/c/17729/2/tests/custom_cluster/test_query_retries.py@760
PS2, Line 760: select count(*) from tpch_parquet.lineitem
> nit. Test robustness. Wonder if this query should be relatively complex (su
Not need. We use the debug action to delay and have enough time to do close_query()


http://gerrit.cloudera.org:8080/#/c/17729/2/tests/custom_cluster/test_query_retries.py@764
PS2, Line 764: time.sleep(1)
             :     # close the query
> nit. May add a comment "close the query" here.
Done


http://gerrit.cloudera.org:8080/#/c/17729/2/tests/custom_cluster/test_query_retries.py@773
PS2, Line 773: 
> nit. should we check all remaining impalads are running instead of the 1st 
This bug only crashes the coordinator, so check the first one is enough



-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 3
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Wed, 28 Jul 2021 06:15:13 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Qifan Chen (Code Review)" <ge...@cloudera.org>.
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 4: Code-Review+1

(2 comments)

Looks great!

http://gerrit.cloudera.org:8080/#/c/17729/2/tests/custom_cluster/test_query_retries.py
File tests/custom_cluster/test_query_retries.py:

http://gerrit.cloudera.org:8080/#/c/17729/2/tests/custom_cluster/test_query_retries.py@760
PS2, Line 760: select count(*) from tpch_parquet.lineitem
> Not need. We use the debug action to delay and have enough time to do close
Done


http://gerrit.cloudera.org:8080/#/c/17729/2/tests/custom_cluster/test_query_retries.py@773
PS2, Line 773: 
> This bug only crashes the coordinator, so check the first one is enough
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 4
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Wed, 28 Jul 2021 13:17:24 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7373/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 5
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Thu, 05 Aug 2021 07:04:26 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 7: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7382/


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 7
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Wed, 11 Aug 2021 06:27:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................

IMPALA-10825: fix impalad crashes when closing the retrying query

The crash happens when canceling the retrying query in web UI.
The canceling action will call ImpalaServer#UnregisterQuery.
The QueryDriver will be null if the query has already been unregistered.

Testing:
 Add test in tests/custom_cluster/test_query_retries.py and manually
tested 100 times to make sure that there was no Impalad crash

Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Reviewed-on: http://gerrit.cloudera.org:8080/17729
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/runtime/query-driver.cc
M tests/custom_cluster/test_query_retries.py
2 files changed, 36 insertions(+), 3 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 9
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 5: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 5
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Thu, 05 Aug 2021 07:04:25 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Kurt Deschler (Code Review)" <ge...@cloudera.org>.
Kurt Deschler has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/17729/2/be/src/runtime/query-driver.cc
File be/src/runtime/query-driver.cc:

http://gerrit.cloudera.org:8080/#/c/17729/2/be/src/runtime/query-driver.cc@190
PS2, Line 190:       return;
Should the MarkAsRetrying() call above be skipped or reset in this case?



-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 2
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 27 Jul 2021 12:08:01 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 7: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 7
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Wed, 11 Aug 2021 00:22:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7380/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 6
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Tue, 10 Aug 2021 03:32:18 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9197/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 3
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Wed, 28 Jul 2021 06:32:45 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Qifan Chen (Code Review)" <ge...@cloudera.org>.
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 2:

(3 comments)

Looks good!

http://gerrit.cloudera.org:8080/#/c/17729/2/tests/custom_cluster/test_query_retries.py
File tests/custom_cluster/test_query_retries.py:

http://gerrit.cloudera.org:8080/#/c/17729/2/tests/custom_cluster/test_query_retries.py@760
PS2, Line 760: select count(*) from tpch_parquet.lineitem
nit. Test robustness. Wonder if this query should be relatively complex (such as a self join of store_sales table) so that its execution time is long enough (say 10s) to allow the close_query() action below to succeed always.


http://gerrit.cloudera.org:8080/#/c/17729/2/tests/custom_cluster/test_query_retries.py@764
PS2, Line 764: time.sleep(1)
             :     self.client.close_query(handle)
nit. May add a comment "close the query" here.


http://gerrit.cloudera.org:8080/#/c/17729/2/tests/custom_cluster/test_query_retries.py@773
PS2, Line 773: first_impalad()
nit. should we check all remaining impalads are running instead of the 1st one only?



-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 2
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 27 Jul 2021 13:30:02 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 4: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 4
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Fri, 30 Jul 2021 08:50:55 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 6: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 6
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Wed, 11 Aug 2021 00:21:21 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 8: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 8
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Thu, 12 Aug 2021 01:38:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10825: fix impalad crashes when closing the retrying query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17729 )

Change subject: IMPALA-10825: fix impalad crashes when closing the retrying query
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7358/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/17729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a2cccbfbdca00b099e0f8d5f2d4bcb4d0a8c3
Gerrit-Change-Number: 17729
Gerrit-PatchSet: 4
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>
Gerrit-Comment-Date: Fri, 30 Jul 2021 02:31:20 +0000
Gerrit-HasComments: No