You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hawq.apache.org by Hubert Zhang <hz...@pivotal.io> on 2016/06/28 06:11:56 UTC
How to know the current status in
Hi all,
I'm running concurrent queries on a 5 nodes cluster, how can I check the
current available resources and total resources?
--
Thanks
Hubert Zhang
Re: How to know the current status in
Posted by Wen Lin <wl...@pivotal.io>.
That means one queue is running.
For virtual segment number, we can know that the quota of one virtual
segment is 256MB, column vsegresourcequota in pg_resqueue catalog table.
From pg_resqueue_status we can know that inusemem is 256MB. So, for this
query, one virtual segment is used.
On Tue, Jun 28, 2016 at 3:02 PM, Hubert Zhang <hz...@pivotal.io> wrote:
> Thanks, wen.
> The final resqueue status in your example, rsqholders is changed from 0 to
> 1.
> Does is mean there is one query running on this queue or only one virtual
> segment being used on this queue? Because a query may using multiple
> virtual segments.
>
>
> On Tue, Jun 28, 2016 at 2:53 PM, Wen Lin <wl...@pivotal.io> wrote:
>
>> Hi,
>>
>> I'd like to use an example to answer it.
>> Suppose hawq is running on Yarn mode.
>> There is a master and 3 segment nodes, each nodes has 8GB memory, 8
>> vcores. So total of the whole cluster is: 24GB memory, 24 vcores.
>> Two resource queues are defined in hadoop yarn. One is for hawq. Another
>> is for other hadoop application. And hawq can use 50% resource of the cluster(configured
>> in capacity-scheduler.xml), which is 12GB memory, 12vcores.
>> [image: Inline image 1]
>> [image: Inline image 2]
>>
>> Then create a resource queue in hawq.
>> [image: Inline image 3]
>>
>> [image: Inline image 4]
>> So in pg_resqueue_status catalog table, we can know the total resources
>> from pg_root queue. Total memory = segsizemax X segmem, 256Mb X 48 = 12GB,
>> Total vcore = segsizemax X segcore, 0.25 X 48 = 24.
>>
>> Then we run a query and check pg_resqueue_status again, we can know the
>> in-use resource, column inusemem and inusecore can show it.
>> [image: Inline image 5]
>>
>> Regards!
>> Lin Wen
>>
>>
>> On Tue, Jun 28, 2016 at 2:11 PM, Hubert Zhang <hz...@pivotal.io> wrote:
>>
>>> Hi all,
>>> I'm running concurrent queries on a 5 nodes cluster, how can I check the
>>> current available resources and total resources?
>>>
>>> --
>>> Thanks
>>>
>>> Hubert Zhang
>>>
>>
>>
>
>
> --
> Thanks
>
> Hubert Zhang
>
Re: How to know the current status in
Posted by Hubert Zhang <hz...@pivotal.io>.
Thanks, wen.
The final resqueue status in your example, rsqholders is changed from 0 to
1.
Does is mean there is one query running on this queue or only one virtual
segment being used on this queue? Because a query may using multiple
virtual segments.
On Tue, Jun 28, 2016 at 2:53 PM, Wen Lin <wl...@pivotal.io> wrote:
> Hi,
>
> I'd like to use an example to answer it.
> Suppose hawq is running on Yarn mode.
> There is a master and 3 segment nodes, each nodes has 8GB memory, 8
> vcores. So total of the whole cluster is: 24GB memory, 24 vcores.
> Two resource queues are defined in hadoop yarn. One is for hawq. Another
> is for other hadoop application. And hawq can use 50% resource of the cluster(configured
> in capacity-scheduler.xml), which is 12GB memory, 12vcores.
> [image: Inline image 1]
> [image: Inline image 2]
>
> Then create a resource queue in hawq.
> [image: Inline image 3]
>
> [image: Inline image 4]
> So in pg_resqueue_status catalog table, we can know the total resources
> from pg_root queue. Total memory = segsizemax X segmem, 256Mb X 48 = 12GB,
> Total vcore = segsizemax X segcore, 0.25 X 48 = 24.
>
> Then we run a query and check pg_resqueue_status again, we can know the
> in-use resource, column inusemem and inusecore can show it.
> [image: Inline image 5]
>
> Regards!
> Lin Wen
>
>
> On Tue, Jun 28, 2016 at 2:11 PM, Hubert Zhang <hz...@pivotal.io> wrote:
>
>> Hi all,
>> I'm running concurrent queries on a 5 nodes cluster, how can I check the
>> current available resources and total resources?
>>
>> --
>> Thanks
>>
>> Hubert Zhang
>>
>
>
--
Thanks
Hubert Zhang
Re: How to know the current status in
Posted by Wen Lin <wl...@pivotal.io>.
Hi,
I'd like to use an example to answer it.
Suppose hawq is running on Yarn mode.
There is a master and 3 segment nodes, each nodes has 8GB memory, 8 vcores.
So total of the whole cluster is: 24GB memory, 24 vcores.
Two resource queues are defined in hadoop yarn. One is for hawq. Another is
for other hadoop application. And hawq can use 50% resource of the
cluster(configured
in capacity-scheduler.xml), which is 12GB memory, 12vcores.
[image: Inline image 1]
[image: Inline image 2]
Then create a resource queue in hawq.
[image: Inline image 3]
[image: Inline image 4]
So in pg_resqueue_status catalog table, we can know the total resources
from pg_root queue. Total memory = segsizemax X segmem, 256Mb X 48 = 12GB,
Total vcore = segsizemax X segcore, 0.25 X 48 = 24.
Then we run a query and check pg_resqueue_status again, we can know the
in-use resource, column inusemem and inusecore can show it.
[image: Inline image 5]
Regards!
Lin Wen
On Tue, Jun 28, 2016 at 2:11 PM, Hubert Zhang <hz...@pivotal.io> wrote:
> Hi all,
> I'm running concurrent queries on a 5 nodes cluster, how can I check the
> current available resources and total resources?
>
> --
> Thanks
>
> Hubert Zhang
>