Re: Increase number of segments per host

[Dominik Choma <do...@gmail.com>]

*Thanks all!*

It turns out that some of my source tables where build with param
default_hash_table_bucket_number set to 6
I maganged to increase parallelism level  by rebuilding source tables with
higher number of hash buckets.

Dominik


2016-09-29 20:14 GMT+02:00 Luis Macedo <lm...@pivotal.io>:

> Dominic,
>
> You can try playing with this GUC: SET enforce_virtual_segment_number =xx
>
> If you are the only used on the box you can set it up to the same number
> of cores available.
>
> More than that will degrade your performance.
>
> Thanks,
>
>
>
> *Luis Macedo | Sr Platform Architect | **Pivotal Inc *
>
> *Mobile:* +55 11 97616-6438
> *Pivotal.io <http://pivotal.io>*
> *Take care of the customers and the rest takes care of itself*
>
> 2016-09-29 12:12 GMT-03:00 Jim Campbell <ja...@pivotal.io>:
>
>> There several GUC settings you can set at a session or query level that
>> can change the number of executors.
>>
>> Other factors to look at are things like table type and how you set up
>> YARN integration.  How is your table distribution set up or table
>> partioning if any?
>>
>> I won't be back to my office until Monday otherwise I could send you
>> additional information.
>>
>>
>> On Thursday, September 29, 2016, Muji Kadry <mu...@gmail.com>
>> wrote:
>>
>>> Hi Dominik,
>>>
>>> You can increase the parallelism by increasing the virtual segment limit
>>> and segment memory for the cluster (using Ambari) and for each query
>>> example below.
>>>
>>> postgres=# set hawq_rm_stmt_vseg_memory='256mb';SETpostgres=# set hawq_rm_stmt_nvseg=10;SETpostgres=# create table t(i integer);CREATE TABLEpostgres=# insert into t values(1);INSERT 0 1
>>>
>>>
>>>
>>> ​Regards,
>>> Muji
>>>
>>> On Thu, Sep 29, 2016 at 8:12 AM, Dominik Choma <do...@gmail.com>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I want to increase parallelism level and hardware utilization  by
>>>> increasing number of segments per host. In PHD it can be done via gpexpand
>>>> http://pivotalhd-210.docs.pivotal.io/doc/2010/ExpandingtheH
>>>> AWQSystem.html#ExpandingtheHAWQSystem-IncreasingSegmentsPerHost.
>>>> Does HAWQ have some utility similar to gpexpand   ?
>>>>
>>>> Dominik
>>>>
>>>>
>>>
>>
>> --
>> Sincerely,
>>
>> Jim Campbell
>> jacampbell@pivotal.io
>>
>>
>