[jira] [Commented] (IMPALA-8825) Add additional counters to PlanRootSink

["Sahil Takiar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-8825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16899020#comment-16899020 ] 

Sahil Takiar commented on IMPALA-8825:
--------------------------------------

Would be nice to have IMPALA-7551 fixed as part of this as well, so linking the two.

> Add additional counters to PlanRootSink
> ---------------------------------------
>
>                 Key: IMPALA-8825
>                 URL: https://issues.apache.org/jira/browse/IMPALA-8825
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Backend
>            Reporter: Sahil Takiar
>            Assignee: Sahil Takiar
>            Priority: Major
>
> The current entry in the runtime profile for {{PLAN_ROOT_SINK}} does not contain much useful information:
> {code:java}
> PLAN_ROOT_SINK:(Total: 234.996ms, non-child: 234.996ms, % non-child: 100.00%)
>     - PeakMemoryUsage: 0{code}
> There are several additional counters we could add to the {{PlanRootSink}} (either the {{BufferedPlanRootSink}} or {{BlockingPlanRootSink}}):
>  * Amount of time spent blocking inside the {{PlanRootSink}} - both the time spent by the client thread waiting for rows to become available and the time spent by the impala thread waiting for the client to consume rows
>  ** So similar to the {{RowBatchQueueGetWaitTime}} and {{RowBatchQueuePutWaitTime}} inside the scan nodes
>  ** The difference between these counters and the ones in {{ClientRequestState}} (e.g. {{ClientFetchWaitTimer}} and {{RowMaterializationTimer}}) should be documented
>  * For {{BufferedPlanRootSink}} there are already several {{Buffer pool}} counters, we should make sure they are exposed in the {{PLAN_ROOT_SINK}} section
>  * Track the number of rows sent (e.g. rows sent to {{PlanRootSink::Send}} and the number of rows fetched (might need to be tracked in the {{ClientRequestState}})
>  ** For {{BlockingPlanRootSink}} the sent and fetched values should be pretty much the same, but for {{BufferedPlanRootSink}} this is more useful
>  ** Similar to {{RowsReturned}} in each exec node
>  * The rate at which rows are sent and fetched
>  ** Should be useful when attempting to debug perf of the fetching rows (e.g. if the send rate is much higher than the fetch rate, then maybe there is something wrong with the client)
>  ** Similar to {{RowsReturnedRate}} in each exec node
> Open to other suggestions for counters that folks think are useful.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org