You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/01/06 00:05:00 UTC
[jira] [Commented] (IMPALA-6101) DataStreamMgr::Cancel() should
take a query ID instead of a finst ID
[ https://issues.apache.org/jira/browse/IMPALA-6101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17259304#comment-17259304 ]
ASF subversion and git services commented on IMPALA-6101:
---------------------------------------------------------
Commit 868a01dca9071978f482aec2a9c3e18aca957914 in impala's branch refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=868a01d ]
IMPALA-6101: call DataStreamMgr::Cancel() once per query
This is a bit of cleanup left over from the KRPC work that could avoid
some lock contention for queries with large numbers of fragments.
The change is just to do cancellation of receivers once per query
instead of once per fragment.
Change-Id: I7677d21f0aaddc3d4b56f72c0470ea850e34611e
Reviewed-on: http://gerrit.cloudera.org:8080/16901
Reviewed-by: Thomas Tauber-Marshall <tm...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> DataStreamMgr::Cancel() should take a query ID instead of a finst ID
> --------------------------------------------------------------------
>
> Key: IMPALA-6101
> URL: https://issues.apache.org/jira/browse/IMPALA-6101
> Project: IMPALA
> Issue Type: Sub-task
> Components: Distributed Exec
> Affects Versions: Impala 2.10.0
> Reporter: Sailesh Mukil
> Assignee: Tim Armstrong
> Priority: Major
> Labels: performance, ramp-up, refactor
>
> Currently our DataStreamMgr class is a singleton class, which means it has a holistic view of all fragment instances running on a given impalad node.
> We currently call DataStreamMgr::Cancel() per fragment instance, which means on every Cancel() call, we do a O(N) lookup through a receiver map maintained by the DataStreamMgr, even though we know that if one fragment instance belonging to a query is cancelled, all other fragment instances belonging to that query will be cancelled as well.
> Instead, if we take a query ID, we can cancel all fragment instances belonging to that query at once, with a single O(N) lookup of the receiver map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org