You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hawq.apache.org by yanqing weng <we...@gmail.com> on 2016/07/01 01:22:09 UTC
elastic execution engine question
Hi HAWQ developer,
Elastic execution engine is the important feature in HAWQ 2.0.
I have a question for this feature in the HAWQ debugging. Is there possible
to control the virtual segment number when run query ? If so, which GUC or
parameter I should set ? Thanks.
Re: elastic execution engine question
Posted by Hubert Zhang <hz...@pivotal.io>.
This GUC hawq_rm_stmt_nvseg is used by resource manager.
Yi, could you please give some detailed information?
On Fri, Jul 1, 2016 at 12:48 PM, Vineet Goel <vg...@pivotal.io> wrote:
> What about hawq_rm_stmt_nvseg ?
>
>
> http://hdb.docs.pivotal.io/20/reference/guc/parameter_definitions.html#hawq_rm_stmt_nvseg
>
> -Vineet
>
>
>
> On Jun 30, 2016, at 6:43 PM, Hubert Zhang <hz...@pivotal.io> wrote:
>
> You can try to the GUC
>
> enforce_virtual_segment_number.
>
> It works for queries without UDF.
>
> On Fri, Jul 1, 2016 at 9:40 AM, yanqing weng <we...@gmail.com>
> wrote:
>
> > Thanks Hubert.
> >
> > Can it be set only use one virtual segment for the query ? Because this
> > can debug query more easily.
> >
> >
> > On Fri, Jul 1, 2016 at 9:35 AM, Hubert Zhang <hz...@pivotal.io> wrote:
> >
> >> For hash tables, The number of virtual segments is equal to the bucket
> >> number of the table. Bucket number can be specified when creating the
> table.
> >>
> >> On Fri, Jul 1, 2016 at 9:31 AM, Ivan Weng <iw...@pivotal.io> wrote:
> >>
> >>> How about hash table?
> >>>
> >>> On Fri, Jul 1, 2016 at 9:30 AM, Hubert Zhang <hz...@pivotal.io>
> wrote:
> >>>
> >>>> For random table, the virtual segment number is related to the data
> >>> size of
> >>>> the table. For large tables, we prefer use more virtual segments to
> >>> achieve
> >>>> better performance. While for small tables, less virtual segments are
> >>>> needed, which could save resources and enable more concurrent queries.
> >>>
> >>>>
> >>>> On Fri, Jul 1, 2016 at 9:22 AM, yanqing weng <we...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Hi HAWQ developer,
> >>>>>
> >>>>> Elastic execution engine is the important feature in HAWQ 2.0.
> >>>>>
> >>>>> I have a question for this feature in the HAWQ debugging. Is there
> >>>> possible
> >>>>> to control the virtual segment number when run query ? If so, which
> >>> GUC
> >>>> or
> >>>>> parameter I should set ? Thanks.
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Thanks
> >>>>
> >>>> Hubert Zhang
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> Thanks
> >>
> >> Hubert Zhang
> >>
> >
> >
>
>
> --
> Thanks
>
> Hubert Zhang
>
>
--
Thanks
Hubert Zhang
Re: elastic execution engine question
Posted by Vineet Goel <vg...@pivotal.io>.
What about hawq_rm_stmt_nvseg ?
http://hdb.docs.pivotal.io/20/reference/guc/parameter_definitions.html#hawq_rm_stmt_nvseg
-Vineet
On Jun 30, 2016, at 6:43 PM, Hubert Zhang <hz...@pivotal.io> wrote:
You can try to the GUC
enforce_virtual_segment_number.
It works for queries without UDF.
On Fri, Jul 1, 2016 at 9:40 AM, yanqing weng <we...@gmail.com> wrote:
> Thanks Hubert.
>
> Can it be set only use one virtual segment for the query ? Because this
> can debug query more easily.
>
>
> On Fri, Jul 1, 2016 at 9:35 AM, Hubert Zhang <hz...@pivotal.io> wrote:
>
>> For hash tables, The number of virtual segments is equal to the bucket
>> number of the table. Bucket number can be specified when creating the table.
>>
>> On Fri, Jul 1, 2016 at 9:31 AM, Ivan Weng <iw...@pivotal.io> wrote:
>>
>>> How about hash table?
>>>
>>> On Fri, Jul 1, 2016 at 9:30 AM, Hubert Zhang <hz...@pivotal.io> wrote:
>>>
>>>> For random table, the virtual segment number is related to the data
>>> size of
>>>> the table. For large tables, we prefer use more virtual segments to
>>> achieve
>>>> better performance. While for small tables, less virtual segments are
>>>> needed, which could save resources and enable more concurrent queries.
>>>
>>>>
>>>> On Fri, Jul 1, 2016 at 9:22 AM, yanqing weng <we...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi HAWQ developer,
>>>>>
>>>>> Elastic execution engine is the important feature in HAWQ 2.0.
>>>>>
>>>>> I have a question for this feature in the HAWQ debugging. Is there
>>>> possible
>>>>> to control the virtual segment number when run query ? If so, which
>>> GUC
>>>> or
>>>>> parameter I should set ? Thanks.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Thanks
>>>>
>>>> Hubert Zhang
>>>>
>>>
>>
>>
>>
>> --
>> Thanks
>>
>> Hubert Zhang
>>
>
>
--
Thanks
Hubert Zhang
Re: elastic execution engine question
Posted by yanqing weng <we...@gmail.com>.
Cool, your help is greatly appreciated.
On Fri, Jul 1, 2016 at 9:43 AM, Hubert Zhang <hz...@pivotal.io> wrote:
> You can try to the GUC
>
> enforce_virtual_segment_number.
>
> It works for queries without UDF.
>
> On Fri, Jul 1, 2016 at 9:40 AM, yanqing weng <we...@gmail.com>
> wrote:
>
>> Thanks Hubert.
>>
>> Can it be set only use one virtual segment for the query ? Because this
>> can debug query more easily.
>>
>>
>> On Fri, Jul 1, 2016 at 9:35 AM, Hubert Zhang <hz...@pivotal.io> wrote:
>>
>>> For hash tables, The number of virtual segments is equal to the bucket
>>> number of the table. Bucket number can be specified when creating the table.
>>>
>>> On Fri, Jul 1, 2016 at 9:31 AM, Ivan Weng <iw...@pivotal.io> wrote:
>>>
>>>> How about hash table?
>>>>
>>>> On Fri, Jul 1, 2016 at 9:30 AM, Hubert Zhang <hz...@pivotal.io> wrote:
>>>>
>>>> > For random table, the virtual segment number is related to the data
>>>> size of
>>>> > the table. For large tables, we prefer use more virtual segments to
>>>> achieve
>>>> > better performance. While for small tables, less virtual segments are
>>>> > needed, which could save resources and enable more concurrent queries.
>>>>
>>>> >
>>>> > On Fri, Jul 1, 2016 at 9:22 AM, yanqing weng <we...@gmail.com>
>>>> > wrote:
>>>> >
>>>> > > Hi HAWQ developer,
>>>> > >
>>>> > > Elastic execution engine is the important feature in HAWQ 2.0.
>>>> > >
>>>> > > I have a question for this feature in the HAWQ debugging. Is there
>>>> > possible
>>>> > > to control the virtual segment number when run query ? If so, which
>>>> GUC
>>>> > or
>>>> > > parameter I should set ? Thanks.
>>>> > >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Thanks
>>>> >
>>>> > Hubert Zhang
>>>> >
>>>>
>>>
>>>
>>>
>>> --
>>> Thanks
>>>
>>> Hubert Zhang
>>>
>>
>>
>
>
> --
> Thanks
>
> Hubert Zhang
>
Re: elastic execution engine question
Posted by Hubert Zhang <hz...@pivotal.io>.
You can try to the GUC
enforce_virtual_segment_number.
It works for queries without UDF.
On Fri, Jul 1, 2016 at 9:40 AM, yanqing weng <we...@gmail.com> wrote:
> Thanks Hubert.
>
> Can it be set only use one virtual segment for the query ? Because this
> can debug query more easily.
>
>
> On Fri, Jul 1, 2016 at 9:35 AM, Hubert Zhang <hz...@pivotal.io> wrote:
>
>> For hash tables, The number of virtual segments is equal to the bucket
>> number of the table. Bucket number can be specified when creating the table.
>>
>> On Fri, Jul 1, 2016 at 9:31 AM, Ivan Weng <iw...@pivotal.io> wrote:
>>
>>> How about hash table?
>>>
>>> On Fri, Jul 1, 2016 at 9:30 AM, Hubert Zhang <hz...@pivotal.io> wrote:
>>>
>>> > For random table, the virtual segment number is related to the data
>>> size of
>>> > the table. For large tables, we prefer use more virtual segments to
>>> achieve
>>> > better performance. While for small tables, less virtual segments are
>>> > needed, which could save resources and enable more concurrent queries.
>>>
>>> >
>>> > On Fri, Jul 1, 2016 at 9:22 AM, yanqing weng <we...@gmail.com>
>>> > wrote:
>>> >
>>> > > Hi HAWQ developer,
>>> > >
>>> > > Elastic execution engine is the important feature in HAWQ 2.0.
>>> > >
>>> > > I have a question for this feature in the HAWQ debugging. Is there
>>> > possible
>>> > > to control the virtual segment number when run query ? If so, which
>>> GUC
>>> > or
>>> > > parameter I should set ? Thanks.
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > Thanks
>>> >
>>> > Hubert Zhang
>>> >
>>>
>>
>>
>>
>> --
>> Thanks
>>
>> Hubert Zhang
>>
>
>
--
Thanks
Hubert Zhang
Re: elastic execution engine question
Posted by yanqing weng <we...@gmail.com>.
Thanks Hubert.
Can it be set only use one virtual segment for the query ? Because this can
debug query more easily.
On Fri, Jul 1, 2016 at 9:35 AM, Hubert Zhang <hz...@pivotal.io> wrote:
> For hash tables, The number of virtual segments is equal to the bucket
> number of the table. Bucket number can be specified when creating the table.
>
> On Fri, Jul 1, 2016 at 9:31 AM, Ivan Weng <iw...@pivotal.io> wrote:
>
>> How about hash table?
>>
>> On Fri, Jul 1, 2016 at 9:30 AM, Hubert Zhang <hz...@pivotal.io> wrote:
>>
>> > For random table, the virtual segment number is related to the data
>> size of
>> > the table. For large tables, we prefer use more virtual segments to
>> achieve
>> > better performance. While for small tables, less virtual segments are
>> > needed, which could save resources and enable more concurrent queries.
>>
>> >
>> > On Fri, Jul 1, 2016 at 9:22 AM, yanqing weng <we...@gmail.com>
>> > wrote:
>> >
>> > > Hi HAWQ developer,
>> > >
>> > > Elastic execution engine is the important feature in HAWQ 2.0.
>> > >
>> > > I have a question for this feature in the HAWQ debugging. Is there
>> > possible
>> > > to control the virtual segment number when run query ? If so, which
>> GUC
>> > or
>> > > parameter I should set ? Thanks.
>> > >
>> >
>> >
>> >
>> > --
>> > Thanks
>> >
>> > Hubert Zhang
>> >
>>
>
>
>
> --
> Thanks
>
> Hubert Zhang
>
Re: elastic execution engine question
Posted by Hubert Zhang <hz...@pivotal.io>.
For hash tables, The number of virtual segments is equal to the bucket
number of the table. Bucket number can be specified when creating the table.
On Fri, Jul 1, 2016 at 9:31 AM, Ivan Weng <iw...@pivotal.io> wrote:
> How about hash table?
>
> On Fri, Jul 1, 2016 at 9:30 AM, Hubert Zhang <hz...@pivotal.io> wrote:
>
> > For random table, the virtual segment number is related to the data size
> of
> > the table. For large tables, we prefer use more virtual segments to
> achieve
> > better performance. While for small tables, less virtual segments are
> > needed, which could save resources and enable more concurrent queries.
> >
> > On Fri, Jul 1, 2016 at 9:22 AM, yanqing weng <we...@gmail.com>
> > wrote:
> >
> > > Hi HAWQ developer,
> > >
> > > Elastic execution engine is the important feature in HAWQ 2.0.
> > >
> > > I have a question for this feature in the HAWQ debugging. Is there
> > possible
> > > to control the virtual segment number when run query ? If so, which GUC
> > or
> > > parameter I should set ? Thanks.
> > >
> >
> >
> >
> > --
> > Thanks
> >
> > Hubert Zhang
> >
>
--
Thanks
Hubert Zhang
Re: elastic execution engine question
Posted by Ivan Weng <iw...@pivotal.io>.
How about hash table?
On Fri, Jul 1, 2016 at 9:30 AM, Hubert Zhang <hz...@pivotal.io> wrote:
> For random table, the virtual segment number is related to the data size of
> the table. For large tables, we prefer use more virtual segments to achieve
> better performance. While for small tables, less virtual segments are
> needed, which could save resources and enable more concurrent queries.
>
> On Fri, Jul 1, 2016 at 9:22 AM, yanqing weng <we...@gmail.com>
> wrote:
>
> > Hi HAWQ developer,
> >
> > Elastic execution engine is the important feature in HAWQ 2.0.
> >
> > I have a question for this feature in the HAWQ debugging. Is there
> possible
> > to control the virtual segment number when run query ? If so, which GUC
> or
> > parameter I should set ? Thanks.
> >
>
>
>
> --
> Thanks
>
> Hubert Zhang
>
Re: elastic execution engine question
Posted by Hubert Zhang <hz...@pivotal.io>.
For random table, the virtual segment number is related to the data size of
the table. For large tables, we prefer use more virtual segments to achieve
better performance. While for small tables, less virtual segments are
needed, which could save resources and enable more concurrent queries.
On Fri, Jul 1, 2016 at 9:22 AM, yanqing weng <we...@gmail.com> wrote:
> Hi HAWQ developer,
>
> Elastic execution engine is the important feature in HAWQ 2.0.
>
> I have a question for this feature in the HAWQ debugging. Is there possible
> to control the virtual segment number when run query ? If so, which GUC or
> parameter I should set ? Thanks.
>
--
Thanks
Hubert Zhang