You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/06/29 15:45:00 UTC
[jira] [Commented] (IMPALA-9829) Add write metrics for Spilling
[ https://issues.apache.org/jira/browse/IMPALA-9829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17147898#comment-17147898 ]
ASF subversion and git services commented on IMPALA-9829:
---------------------------------------------------------
Commit 94c1b6354b9da14db7ea9e59f09fcb06bae2acfe in impala's branch refs/heads/master from Yida Wu
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=94c1b63 ]
IMPALA-9829: Add Write Metrics for Spilling
Currently, we have read metrics for spilling, in this patch, we add
support for write metrics. The new metrics could be useful to measure
the write operations and target performance issues when involving in
spilling to remote disks(S3) (IMPALA-9828).
The metrics added record the information includes:
1. write latency of each write operation to the disk, metric kind:
HistogramMetric, unit: nanosecond.
2. write size of each write operation to the disk, metric kind:
HistogramMetric, unit: Bytes.
3. number of write IO errors when writing to the disk, metric kind:
IntCounter.
Testing:
* added DiskIoMgrTest.MetricsOfWriteSizeAndLatency
* added DiskIoMgrTest.MetricsOfWriteIoError
Ran unit test disk-io-mgr-test and pre-commit test
Change-Id: I152b9c5339cedabe33f8873a2bbf651aa5dbb914
Reviewed-on: http://gerrit.cloudera.org:8080/16083
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Add write metrics for Spilling
> ------------------------------
>
> Key: IMPALA-9829
> URL: https://issues.apache.org/jira/browse/IMPALA-9829
> Project: IMPALA
> Issue Type: Task
> Components: Backend
> Reporter: Abhishek Rawat
> Assignee: Yida Wu
> Priority: Major
> Fix For: Impala 4.0
>
>
> Extend io-mgr metrics to include additional metrics as a requirement from Spilling to S3.
>
> {code:java}
> impala-server.io-mgr.queue-$0.write-latency // Histogram of write operation times on specific disk queue.
> impala-server.io-mgr.queue-$0.write-size // Histogram of write operation sizes on specific disk queue.
> impala-server.io-mgr.queue-$0.write-io-error // I/O errors encountered during writing.
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org