[jira] [Commented] (IMPALA-12176) Improve client fetch metrics

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731035#comment-17731035 ] 

ASF subversion and git services commented on IMPALA-12176:
----------------------------------------------------------

Commit aa164e3cbc490b69403959f979e55b76c3d0e0ca in impala's branch refs/heads/master from Kurt Deschler
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=aa164e3cb ]

IMPALA-12176: Improve client fetch metrics

This patch makes multiple improvements to query profile and RPC metrics
to improve observability and allow more detailed analysis of where time
is being spent by client RPCs.

- A new CreateResultSetTime metric has been added to PLAN_ROOT_SINK node
  in the query profile. This timer isolates the cost to convert fetched
  rows to the client protocol.
- Read/Write time is now tracked during client RPC execution and added to
  the rpcz JSON output. A checkbox in the /rpcz Web UI page enables
  display of the Read/Write stats.
- Read and Write time are defined from Thrift callbacks defined in
  apache::thrift::TProcessorEventHandler. Read time includes reading and
  deserializing Thrift RPC args from the transport. Write time includes
  serializing, writing, and flushing Thrift RPC args to the transport.
- Client RPC cost is tracked on a per-query basis and displayed in the
  server profile as RPCCount, RPCReadTimer, and RPCWriteTimer
- Accuracy of RPC histograms is changed from milliseconds to microseconds

Testing:
tests added to test_fetch.py and test_web_pages.py

Change-Id: I986f3f2afac1775274895393969b270cf956b262
Reviewed-on: http://gerrit.cloudera.org:8080/19966
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Improve client fetch metrics
> ----------------------------
>
>                 Key: IMPALA-12176
>                 URL: https://issues.apache.org/jira/browse/IMPALA-12176
>             Project: IMPALA
>          Issue Type: Improvement
>            Reporter: Kurt Deschler
>            Assignee: Kurt Deschler
>            Priority: Major
>
> These changes address limitations with the current metrics:
>  * ClientFetchWaitTimer includes both Thrift serialization/write and client time.
>  * RowMaterializationTimer includes both coordinator fetch time and time to convert rows to the client protocol.
> Proposed changes:
>  * Add CreateResultSetTime metric to PLAN_ROOT_SINK node in the query profile. This will isolate the cost to convert fetched rows to the client protocol.
>  * Add read/write times for RPCs in /rpcz webui. These will be hidden by default with a checkbox to enable display.
>  * Add sum to RPC histogram metrics
>  * Show client RPC read/write/count stats in the query profile



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org