[jira] [Created] (IMPALA-9471) To monitor the data cache, there are several improvements to metrics that would help. This tracks all the distinct metrics changes.

["Joe McDonnell (Jira)" <ji...@apache.org>]

Joe McDonnell created IMPALA-9471:
-------------------------------------

             Summary: To monitor the data cache, there are several improvements to metrics that would help. This tracks all the distinct metrics changes.
                 Key: IMPALA-9471
                 URL: https://issues.apache.org/jira/browse/IMPALA-9471
             Project: IMPALA
          Issue Type: Epic
          Components: Backend
    Affects Versions: Impala 3.4.0
            Reporter: Joe McDonnell
            Assignee: Joe McDonnell


Currently, the data cache has the following metrics:
{noformat}
impala-server.io-mgr.remote-data-cache-hit-bytes
impala-server.io-mgr.remote-data-cache-miss-bytes
impala-server.io-mgr.remote-data-cache-total-bytes
impala-server.io-mgr.remote-data-cache-dropped-byte{noformat}
There are several questions that these metrics will not answer, especially when we start to consider changes to eviction algorithms. Here are some questions that we may want to be able to answer:
 # How much memory is being used to track metadata?
 # What is the distribution of size of entries in the cache?
 # How many entries are in the cache?
 # What are the hit/miss counts (as opposed to the hit bytes)?
 # What is the actual disk usage (as seen by the OS)?

This is an epic to track adding metrics to answer these questions (and other similar questions).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org