You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/08/06 15:11:00 UTC
[jira] [Commented] (IMPALA-6984) Coordinator should cancel backends
when returning EOS
[ https://issues.apache.org/jira/browse/IMPALA-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17172454#comment-17172454 ]
ASF subversion and git services commented on IMPALA-6984:
---------------------------------------------------------
Commit c413f9b558d51de877f497590baf14139ad5cf99 in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=c413f9b ]
IMPALA-10047: Revert core piece of IMPALA-6984
Performance testing on TPC-DS found a peformance regression
on short queries due to delayed exec status reports. Further
testing traced this back to IMPALA-6984's behavior of
cancelling backends on EOS. The coordinator log show that
CancelBackends() call intermittently taking 10 seconds due
to timing out in the RPC layer.
As a temporary workaround, this reverts the core part of
IMPALA-6984 that added that CancelBackends() call for EOS.
It leaves the rest of IMPALA-6984 intact, as other code has built
on top of it.
Testing:
- Core job
- Performance tests
Change-Id: Ibf00a56e91f0376eaaa552e3bb4763501bfb49e8
(cherry picked from commit b91f3c0e064d592f3cdf2a2e089ca6546133ba55)
Reviewed-on: http://gerrit.cloudera.org:8080/16288
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Coordinator should cancel backends when returning EOS
> -----------------------------------------------------
>
> Key: IMPALA-6984
> URL: https://issues.apache.org/jira/browse/IMPALA-6984
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Affects Versions: Impala 3.0
> Reporter: Daniel Hecht
> Assignee: Tim Armstrong
> Priority: Major
> Labels: query-lifecycle
> Fix For: Impala 4.0
>
>
> Currently, the Coordinator waits for backends rather than proactively cancelling them in the case of hitting EOS. There's a tangled mess that makes it tricky to proactively cancel the backends related to how {{Coordinator::ComputeQuerySummary()}} works – we can't update the summary until the profiles are no longer changing (which also makes sense given that we want the exec summary to be consistent with the final profile). But we current tie together the FIS status and the profile, and cancellation of backends causes the FIS to return CANCELLED, which then means that the remaining FIS on that backend won't produce a final profile.
> With the rework of the protocol for IMPALA-2990 we should make it possible to sort this out such that a final profile can be requested regardless of how a FIS ends execution.
> This also relates to IMPALA-5783.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org