Safe to upgrade to Hive 0.7.0?

[Ryan LeCompte <le...@gmail.com>]

Hey guys,

We would like to experiment with some of the features in the current trunk
(0.7) release. Do you think at this point it's nearing "production-ready" ?

Are there any metastore migration scripts or anything like that before
upgrading to 0.7 from 0.6?

Thanks,
Ryan

Re: Safe to upgrade to Hive 0.7.0?

[Edward Capriolo <ed...@gmail.com>]

On Mon, Feb 7, 2011 at 7:27 PM, Gokul Pillai <go...@gmail.com> wrote:
> I have a Hadoop 0.20 cluster with Hive 0.6.
> I want to try out the ngrams functions in 0.7 (trunk) and so want to
> upgrade.
> What would you recommend. Can I have a Hive 0.7 install in a different
> location (for trial) and point it to the same cluster.
> If not, is there an upgrade utility?
> I am not concerned about existing tables. I just want to make sure I can run
> Hive 0.6 and Hive 0.7 side by side.
> Regards
> Gokul
>
> On Wed, Dec 8, 2010 at 9:26 AM, Edward Capriolo <ed...@gmail.com>
> wrote:
>>
>> On Wed, Dec 8, 2010 at 12:06 PM, Ning Zhang <nz...@fb.com> wrote:
>> > These are datanucleus paramesters found in
>> > hive-default.xml/hive-site.xml. If you change them to the following the
>> > metastore schema won't be changed automatically (may be good for preventing
>> > accidents).
>> >
>> >
>> > <property>
>> >  <name>datanucleus.autoCreateSchema</name>
>> >  <value>false</value>
>> > </property>
>> >
>> > <property>
>> >  <name>datanucleus.fixedDatastore</name>
>> >  <value>true</value>
>> > </property>
>> >
>> >
>> > ________________________________________
>> > From: Edward Capriolo [edlinuxguru@gmail.com]
>> > Sent: Wednesday, December 08, 2010 7:19 AM
>> > To: user@hive.apache.org
>> > Cc: <hi...@hadoop.apache.org>
>> > Subject: Re: Safe to upgrade to Hive 0.7.0?
>> >
>> > On Wed, Dec 8, 2010 at 9:55 AM, Ryan LeCompte <le...@gmail.com>
>> > wrote:
>> >> Hey guys,
>> >> We would like to experiment with some of the features in the current
>> >> trunk
>> >> (0.7) release. Do you think at this point it's nearing
>> >> "production-ready" ?
>> >> Are there any metastore migration scripts or anything like that before
>> >> upgrading to 0.7 from 0.6?
>> >> Thanks,
>> >> Ryan
>> >>
>> >
>> > Ryan,
>> >
>> > This is hard to say. There are many changes going in around indexing
>> > and authorization. IMHO everyone is taking a lot of care in the design
>> > however, it is still subject to change.
>> >
>> > If i were compelled to update I would spin up a new instance into a
>> > new meta-store back end. Then compare the schema of current with 0.6.X
>> > like a diff of 'show create tables'.
>> >
>> > Just an FYI. I applied two different hive trunks between 0.5 and 0.6.
>> > and I never needed the update scripts because hive happily updated the
>> > schema for me. After my final upgrade to 0.7.0 I re-ran everything in
>> > the upgrade script just to be safe. Nothing noticeably bad happened to
>> > me during the duration.
>> >
>> > I suspect many people will not need to run the update scripts since in
>> > the stock configuration hive seems to auto-update anyway. (Someone
>> > correct me if I am wrong here.) Does anyone know how to specifically
>> > which variable controls this?
>> >
>> > Edward
>> >
>>
>> https://issues.apache.org/jira/browse/HIVE-1841 -- please comment
>
>

I would not do it. The new version will attempt to update the
meta-store. I would just attempt to build n grams udf into a separate
JAR and load it up with create temporary function.

Re: Safe to upgrade to Hive 0.7.0?

[Gokul Pillai <go...@gmail.com>]

I have a Hadoop 0.20 cluster with Hive 0.6.
I want to try out the ngrams functions in 0.7 (trunk) and so want to
upgrade.

What would you recommend. Can I have a Hive 0.7 install in a different
location (for trial) and point it to the same cluster.
If not, is there an upgrade utility?

I am not concerned about existing tables. I just want to make sure I can run
Hive 0.6 and Hive 0.7 side by side.

Regards
Gokul

On Wed, Dec 8, 2010 at 9:26 AM, Edward Capriolo <ed...@gmail.com>wrote:

> On Wed, Dec 8, 2010 at 12:06 PM, Ning Zhang <nz...@fb.com> wrote:
> > These are datanucleus paramesters found in
> hive-default.xml/hive-site.xml. If you change them to the following the
> metastore schema won't be changed automatically (may be good for preventing
> accidents).
> >
> >
> > <property>
> >  <name>datanucleus.autoCreateSchema</name>
> >  <value>false</value>
> > </property>
> >
> > <property>
> >  <name>datanucleus.fixedDatastore</name>
> >  <value>true</value>
> > </property>
> >
> >
> > ________________________________________
> > From: Edward Capriolo [edlinuxguru@gmail.com]
> > Sent: Wednesday, December 08, 2010 7:19 AM
> > To: user@hive.apache.org
> > Cc: <hi...@hadoop.apache.org>
> > Subject: Re: Safe to upgrade to Hive 0.7.0?
> >
> > On Wed, Dec 8, 2010 at 9:55 AM, Ryan LeCompte <le...@gmail.com>
> wrote:
> >> Hey guys,
> >> We would like to experiment with some of the features in the current
> trunk
> >> (0.7) release. Do you think at this point it's nearing
> "production-ready" ?
> >> Are there any metastore migration scripts or anything like that before
> >> upgrading to 0.7 from 0.6?
> >> Thanks,
> >> Ryan
> >>
> >
> > Ryan,
> >
> > This is hard to say. There are many changes going in around indexing
> > and authorization. IMHO everyone is taking a lot of care in the design
> > however, it is still subject to change.
> >
> > If i were compelled to update I would spin up a new instance into a
> > new meta-store back end. Then compare the schema of current with 0.6.X
> > like a diff of 'show create tables'.
> >
> > Just an FYI. I applied two different hive trunks between 0.5 and 0.6.
> > and I never needed the update scripts because hive happily updated the
> > schema for me. After my final upgrade to 0.7.0 I re-ran everything in
> > the upgrade script just to be safe. Nothing noticeably bad happened to
> > me during the duration.
> >
> > I suspect many people will not need to run the update scripts since in
> > the stock configuration hive seems to auto-update anyway. (Someone
> > correct me if I am wrong here.) Does anyone know how to specifically
> > which variable controls this?
> >
> > Edward
> >
>
> https://issues.apache.org/jira/browse/HIVE-1841 -- please comment
>

Re: Safe to upgrade to Hive 0.7.0?

[Edward Capriolo <ed...@gmail.com>]

On Wed, Dec 8, 2010 at 12:06 PM, Ning Zhang <nz...@fb.com> wrote:
> These are datanucleus paramesters found in hive-default.xml/hive-site.xml. If you change them to the following the metastore schema won't be changed automatically (may be good for preventing accidents).
>
>
> <property>
>  <name>datanucleus.autoCreateSchema</name>
>  <value>false</value>
> </property>
>
> <property>
>  <name>datanucleus.fixedDatastore</name>
>  <value>true</value>
> </property>
>
>
> ________________________________________
> From: Edward Capriolo [edlinuxguru@gmail.com]
> Sent: Wednesday, December 08, 2010 7:19 AM
> To: user@hive.apache.org
> Cc: <hi...@hadoop.apache.org>
> Subject: Re: Safe to upgrade to Hive 0.7.0?
>
> On Wed, Dec 8, 2010 at 9:55 AM, Ryan LeCompte <le...@gmail.com> wrote:
>> Hey guys,
>> We would like to experiment with some of the features in the current trunk
>> (0.7) release. Do you think at this point it's nearing "production-ready" ?
>> Are there any metastore migration scripts or anything like that before
>> upgrading to 0.7 from 0.6?
>> Thanks,
>> Ryan
>>
>
> Ryan,
>
> This is hard to say. There are many changes going in around indexing
> and authorization. IMHO everyone is taking a lot of care in the design
> however, it is still subject to change.
>
> If i were compelled to update I would spin up a new instance into a
> new meta-store back end. Then compare the schema of current with 0.6.X
> like a diff of 'show create tables'.
>
> Just an FYI. I applied two different hive trunks between 0.5 and 0.6.
> and I never needed the update scripts because hive happily updated the
> schema for me. After my final upgrade to 0.7.0 I re-ran everything in
> the upgrade script just to be safe. Nothing noticeably bad happened to
> me during the duration.
>
> I suspect many people will not need to run the update scripts since in
> the stock configuration hive seems to auto-update anyway. (Someone
> correct me if I am wrong here.) Does anyone know how to specifically
> which variable controls this?
>
> Edward
>

https://issues.apache.org/jira/browse/HIVE-1841 -- please comment

RE: Safe to upgrade to Hive 0.7.0?

[Ning Zhang <nz...@fb.com>]

These are datanucleus paramesters found in hive-default.xml/hive-site.xml. If you change them to the following the metastore schema won't be changed automatically (may be good for preventing accidents).


<property>
  <name>datanucleus.autoCreateSchema</name>
  <value>false</value>
</property>

<property>
  <name>datanucleus.fixedDatastore</name>
  <value>true</value>
</property>


________________________________________
From: Edward Capriolo [edlinuxguru@gmail.com]
Sent: Wednesday, December 08, 2010 7:19 AM
To: user@hive.apache.org
Cc: <hi...@hadoop.apache.org>
Subject: Re: Safe to upgrade to Hive 0.7.0?

On Wed, Dec 8, 2010 at 9:55 AM, Ryan LeCompte <le...@gmail.com> wrote:
> Hey guys,
> We would like to experiment with some of the features in the current trunk
> (0.7) release. Do you think at this point it's nearing "production-ready" ?
> Are there any metastore migration scripts or anything like that before
> upgrading to 0.7 from 0.6?
> Thanks,
> Ryan
>

Ryan,

This is hard to say. There are many changes going in around indexing
and authorization. IMHO everyone is taking a lot of care in the design
however, it is still subject to change.

If i were compelled to update I would spin up a new instance into a
new meta-store back end. Then compare the schema of current with 0.6.X
like a diff of 'show create tables'.

Just an FYI. I applied two different hive trunks between 0.5 and 0.6.
and I never needed the update scripts because hive happily updated the
schema for me. After my final upgrade to 0.7.0 I re-ran everything in
the upgrade script just to be safe. Nothing noticeably bad happened to
me during the duration.

I suspect many people will not need to run the update scripts since in
the stock configuration hive seems to auto-update anyway. (Someone
correct me if I am wrong here.) Does anyone know how to specifically
which variable controls this?

Edward

Re: Safe to upgrade to Hive 0.7.0?

[Edward Capriolo <ed...@gmail.com>]

On Wed, Dec 8, 2010 at 9:55 AM, Ryan LeCompte <le...@gmail.com> wrote:
> Hey guys,
> We would like to experiment with some of the features in the current trunk
> (0.7) release. Do you think at this point it's nearing "production-ready" ?
> Are there any metastore migration scripts or anything like that before
> upgrading to 0.7 from 0.6?
> Thanks,
> Ryan
>

Ryan,

This is hard to say. There are many changes going in around indexing
and authorization. IMHO everyone is taking a lot of care in the design
however, it is still subject to change.

If i were compelled to update I would spin up a new instance into a
new meta-store back end. Then compare the schema of current with 0.6.X
like a diff of 'show create tables'.

Just an FYI. I applied two different hive trunks between 0.5 and 0.6.
and I never needed the update scripts because hive happily updated the
schema for me. After my final upgrade to 0.7.0 I re-ran everything in
the upgrade script just to be safe. Nothing noticeably bad happened to
me during the duration.

I suspect many people will not need to run the update scripts since in
the stock configuration hive seems to auto-update anyway. (Someone
correct me if I am wrong here.) Does anyone know how to specifically
which variable controls this?

Edward