You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Owen O'Malley <om...@apache.org> on 2012/03/02 17:45:18 UTC

Changing defaults for dynamic partitioning and enforced bucketing?

All,
  Would it make sense to change the default configuration to set
hive.enforce.bucketing and hive.exec.dynamic.partition to true? I find
that I usually change them and I wonder what other peoples' experience
with them is.

Thoughts?
  Owen

RE: Changing defaults for dynamic partitioning and enforced bucketing?

Posted by "Aggarwal, Vaibhav" <va...@amazon.com>.
I am also in favor of setting hive.exec.dynamic.partition to true.

-----Original Message-----
From: Namit Jain [mailto:njain@fb.com] 
Sent: Friday, March 02, 2012 9:03 AM
To: dev@hive.apache.org
Subject: Re: Changing defaults for dynamic partitioning and enforced bucketing?

I agree with changing the hive.exec.dynamic.partitions.


I am not sure about enforcing hive.enforcing.bucketing. It can have a additional cost, which may or may not be needed. If we change this, we should trust the metastore completelely and also change hive.optimize.bucketmapjoin and hive.optimize.bucketmapjoin.sortedmerge to true.

This may break some legacy applications, but it may completely make sense for new setups.


Thanks,
-namit






On 3/2/12 8:45 AM, "Owen O'Malley" <om...@apache.org> wrote:

>All,
>  Would it make sense to change the default configuration to set 
>hive.enforce.bucketing and hive.exec.dynamic.partition to true? I find 
>that I usually change them and I wonder what other peoples' experience 
>with them is.
>
>Thoughts?
>  Owen

Re: Changing defaults for dynamic partitioning and enforced bucketing?

Posted by Namit Jain <nj...@fb.com>.
I agree with changing the hive.exec.dynamic.partitions.


I am not sure about enforcing hive.enforcing.bucketing. It can have a
additional
cost, which may or may not be needed. If we change this, we should trust
the metastore
completelely and also change hive.optimize.bucketmapjoin and
hive.optimize.bucketmapjoin.sortedmerge to true.

This may break some legacy applications, but it may completely make sense
for new setups.


Thanks,
-namit






On 3/2/12 8:45 AM, "Owen O'Malley" <om...@apache.org> wrote:

>All,
>  Would it make sense to change the default configuration to set
>hive.enforce.bucketing and hive.exec.dynamic.partition to true? I find
>that I usually change them and I wonder what other peoples' experience
>with them is.
>
>Thoughts?
>  Owen