You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/11/13 21:59:00 UTC
[jira] [Commented] (IMPALA-7819) Scanners should include "storage
wait time" per scan node
[ https://issues.apache.org/jira/browse/IMPALA-7819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16685788#comment-16685788 ]
ASF subversion and git services commented on IMPALA-7819:
---------------------------------------------------------
Commit 9f81ef1f1c5204d674f00a5c1a33cc859afd4efc in impala's branch refs/heads/master from [~tarmstrong@cloudera.com]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=9f81ef1 ]
IMPALA-7819: add per-node storage wait time to profile
Adds ScannerIoWaitTime and KuduClientTime to measure time spent waiting
for the I/O manager and time spent in the Kudu client, respectively.
To minimise the additional overhead, add support to the timer
infrastructure to update multiple counters with one stopwatch. Apply
this in some other places too.
Testing:
Ran core tests. Ran some queries manually with and without mt_dop
and sanity-checked the profiles.
Change-Id: I74a496efdbe9e804db779c9753b00a303a270580
Reviewed-on: http://gerrit.cloudera.org:8080/11894
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Scanners should include "storage wait time" per scan node
> ---------------------------------------------------------
>
> Key: IMPALA-7819
> URL: https://issues.apache.org/jira/browse/IMPALA-7819
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Fix For: Impala 3.2.0
>
>
> We have TotalStorageWaitTime per fragment instance, which aggregates various things like times spent waiting for the I/O manager and time spent in the Kudu client. This is useful but often we want to understand why a specific scan is slow. This is tricky because TotalStorageWaitTime and the various scanner thread timers in the profile are in different subsections and don't measure comparable things if there are multiple scans in the profile.
> We should also include this timings in the scan profile to address this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org