You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@impala.apache.org by Fawze Abujaber <fa...@gmail.com> on 2018/02/20 00:59:19 UTC
Accumulated impala metrics
Hello All,
I have several impala resource pools and i'm trying to make a monthly usage
report between the pools in %.
Currently i'm collecting 3 metrics: running time, CPU threads memory and
memory accrual.
What do you think the best way to calculate the usage on? i was thinking to
based only on the memory accrual.
Re: Accumulated impala metrics
Posted by Fawze Abujaber <fa...@gmail.com>.
Hi Tim,
As always, thanks for your quick response.
Indeed the HDFS bytes scan can cover the IO resources side, and if we are
talking about resources we always talking about memory, cpu and IO.
To make my calculations simple In calculating the monthly usage I can rely
on the accural memory, yes it will not cover all the cases but I think it
will the 80% of the cases which can be a good indicator.
On Wed, 21 Feb 2018 at 21:47 Tim Armstrong <ta...@cloudera.com> wrote:
> Memory accrual is a good metric. It may also be worth adding HDFS bytes
> scanned, since scan-intensive queries that filter or reduce a lot of data
> can use a lot of resources but not much memory.
>
> On Mon, Feb 19, 2018 at 4:59 PM, Fawze Abujaber <fa...@gmail.com> wrote:
>
>> Hello All,
>>
>> I have several impala resource pools and i'm trying to make a monthly
>> usage report between the pools in %.
>>
>> Currently i'm collecting 3 metrics: running time, CPU threads memory and
>> memory accrual.
>>
>> What do you think the best way to calculate the usage on? i was thinking
>> to based only on the memory accrual.
>>
>>
>>
>
Re: Accumulated impala metrics
Posted by Tim Armstrong <ta...@cloudera.com>.
Memory accrual is a good metric. It may also be worth adding HDFS bytes
scanned, since scan-intensive queries that filter or reduce a lot of data
can use a lot of resources but not much memory.
On Mon, Feb 19, 2018 at 4:59 PM, Fawze Abujaber <fa...@gmail.com> wrote:
> Hello All,
>
> I have several impala resource pools and i'm trying to make a monthly
> usage report between the pools in %.
>
> Currently i'm collecting 3 metrics: running time, CPU threads memory and
> memory accrual.
>
> What do you think the best way to calculate the usage on? i was thinking
> to based only on the memory accrual.
>
>
>