You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Priyam Gupta <pr...@zapr.in> on 2019/12/09 05:37:56 UTC
Hive 1.1.0 support on hive metastore 2.3.0
Hi,
In order to have single hive metastore supporting different clusters of
hive, I have upgraded my hive metastore schema (on postgres) from version
1.1.0 to version 2.3.0. Will hive cluster on version 1.1.0 work if I point
it to upgraded metastore db which is as per hive version 2.3.0.
Thanks,
Priyam
Re: Hive 1.1.0 support on hive metastore 2.3.0
Posted by Priyam Gupta <pr...@zapr.in>.
Thanks. Will try out MR3.
On Mon, Dec 9, 2019 at 3:51 PM Sungwoo Park <gl...@gmail.com> wrote:
> I didn't try to run multiple versions of Hive on the same cluster. If your
> installation of Hive uses Tez installed on the Hadoop system, I guess
> running multiple versions of Hive might not be easy because different
> versions of Hive use different versions of Tez (especially if you want to
> run Hive-LLAP). For me, I use Hive on MR3 (well, because we created MR3).
> You can run all of Hive 1, 2, 3 concurrently on the same cluster. And there
> is no installation process -- just unpack the tar balls and configure
> files, and you are all set.
>
> --- Sungwoo
>
> On Mon, Dec 9, 2019 at 6:27 PM Priyam Gupta <pr...@zapr.in> wrote:
>
>> Thanks Park for sharing the tests that you did. I will try out for the
>> specific version for my use cases.
>>
>> Is there any better way/approach where we can have a single meta store
>> and can launch multiple hive clusters of different versions point to the
>> same metastore db.
>>
>> Thanks.
>>
>>
>>
>>
>> On Mon, Dec 9, 2019 at 12:46 PM Sungwoo Park <gl...@gmail.com> wrote:
>>
>>> Not a definitive answer, but my test result might help. I tested with
>>> HiveServer2 1.2.2 and Metastore 2.3.6. Queries in the TPC-DS benchmark
>>> (which only read data and never update) run okay. Creating new tables and
>>> loading data to tables also work okay. So, I guess for basic uses of Hive,
>>> running HiveServer2 1.2 against Metastore 2.3 should be fine.
>>>
>>> What I am not sure about is whether queries involving transactions work
>>> okay when HiveServer2 1.2 connects to Metastore 2.3. Please note that I
>>> have not tested with Hive 1.1. Perhaps those familiar with internals of
>>> Metastore in this mailing list could give more accurate answers.
>>>
>>> As a side note, HiveServer2 2.3 can connect to Metastore 3.1. However,
>>> HiveServer2 1.2 seems to have a problem with connecting to Metastore 3.1.
>>>
>>> Cheers,
>>>
>>> --- gla
>>>
>>> On Mon, Dec 9, 2019 at 2:38 PM Priyam Gupta <pr...@zapr.in> wrote:
>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>> In order to have single hive metastore supporting different clusters of
>>>> hive, I have upgraded my hive metastore schema (on postgres) from version
>>>> 1.1.0 to version 2.3.0. Will hive cluster on version 1.1.0 work if I point
>>>> it to upgraded metastore db which is as per hive version 2.3.0.
>>>>
>>>>
>>>> Thanks,
>>>> Priyam
>>>>
>>>
Re: Hive 1.1.0 support on hive metastore 2.3.0
Posted by Sungwoo Park <gl...@gmail.com>.
I didn't try to run multiple versions of Hive on the same cluster. If your
installation of Hive uses Tez installed on the Hadoop system, I guess
running multiple versions of Hive might not be easy because different
versions of Hive use different versions of Tez (especially if you want to
run Hive-LLAP). For me, I use Hive on MR3 (well, because we created MR3).
You can run all of Hive 1, 2, 3 concurrently on the same cluster. And there
is no installation process -- just unpack the tar balls and configure
files, and you are all set.
--- Sungwoo
On Mon, Dec 9, 2019 at 6:27 PM Priyam Gupta <pr...@zapr.in> wrote:
> Thanks Park for sharing the tests that you did. I will try out for the
> specific version for my use cases.
>
> Is there any better way/approach where we can have a single meta store and
> can launch multiple hive clusters of different versions point to the same
> metastore db.
>
> Thanks.
>
>
>
>
> On Mon, Dec 9, 2019 at 12:46 PM Sungwoo Park <gl...@gmail.com> wrote:
>
>> Not a definitive answer, but my test result might help. I tested with
>> HiveServer2 1.2.2 and Metastore 2.3.6. Queries in the TPC-DS benchmark
>> (which only read data and never update) run okay. Creating new tables and
>> loading data to tables also work okay. So, I guess for basic uses of Hive,
>> running HiveServer2 1.2 against Metastore 2.3 should be fine.
>>
>> What I am not sure about is whether queries involving transactions work
>> okay when HiveServer2 1.2 connects to Metastore 2.3. Please note that I
>> have not tested with Hive 1.1. Perhaps those familiar with internals of
>> Metastore in this mailing list could give more accurate answers.
>>
>> As a side note, HiveServer2 2.3 can connect to Metastore 3.1. However,
>> HiveServer2 1.2 seems to have a problem with connecting to Metastore 3.1.
>>
>> Cheers,
>>
>> --- gla
>>
>> On Mon, Dec 9, 2019 at 2:38 PM Priyam Gupta <pr...@zapr.in> wrote:
>>
>>>
>>>
>>> Hi,
>>>
>>> In order to have single hive metastore supporting different clusters of
>>> hive, I have upgraded my hive metastore schema (on postgres) from version
>>> 1.1.0 to version 2.3.0. Will hive cluster on version 1.1.0 work if I point
>>> it to upgraded metastore db which is as per hive version 2.3.0.
>>>
>>>
>>> Thanks,
>>> Priyam
>>>
>>
Re: Hive 1.1.0 support on hive metastore 2.3.0
Posted by Priyam Gupta <pr...@zapr.in>.
Thanks Park for sharing the tests that you did. I will try out for the
specific version for my use cases.
Is there any better way/approach where we can have a single meta store and
can launch multiple hive clusters of different versions point to the same
metastore db.
Thanks.
On Mon, Dec 9, 2019 at 12:46 PM Sungwoo Park <gl...@gmail.com> wrote:
> Not a definitive answer, but my test result might help. I tested with
> HiveServer2 1.2.2 and Metastore 2.3.6. Queries in the TPC-DS benchmark
> (which only read data and never update) run okay. Creating new tables and
> loading data to tables also work okay. So, I guess for basic uses of Hive,
> running HiveServer2 1.2 against Metastore 2.3 should be fine.
>
> What I am not sure about is whether queries involving transactions work
> okay when HiveServer2 1.2 connects to Metastore 2.3. Please note that I
> have not tested with Hive 1.1. Perhaps those familiar with internals of
> Metastore in this mailing list could give more accurate answers.
>
> As a side note, HiveServer2 2.3 can connect to Metastore 3.1. However,
> HiveServer2 1.2 seems to have a problem with connecting to Metastore 3.1.
>
> Cheers,
>
> --- gla
>
> On Mon, Dec 9, 2019 at 2:38 PM Priyam Gupta <pr...@zapr.in> wrote:
>
>>
>>
>> Hi,
>>
>> In order to have single hive metastore supporting different clusters of
>> hive, I have upgraded my hive metastore schema (on postgres) from version
>> 1.1.0 to version 2.3.0. Will hive cluster on version 1.1.0 work if I point
>> it to upgraded metastore db which is as per hive version 2.3.0.
>>
>>
>> Thanks,
>> Priyam
>>
>
Re: Hive 1.1.0 support on hive metastore 2.3.0
Posted by Sungwoo Park <gl...@gmail.com>.
Not a definitive answer, but my test result might help. I tested with
HiveServer2 1.2.2 and Metastore 2.3.6. Queries in the TPC-DS benchmark
(which only read data and never update) run okay. Creating new tables and
loading data to tables also work okay. So, I guess for basic uses of Hive,
running HiveServer2 1.2 against Metastore 2.3 should be fine.
What I am not sure about is whether queries involving transactions work
okay when HiveServer2 1.2 connects to Metastore 2.3. Please note that I
have not tested with Hive 1.1. Perhaps those familiar with internals of
Metastore in this mailing list could give more accurate answers.
As a side note, HiveServer2 2.3 can connect to Metastore 3.1. However,
HiveServer2 1.2 seems to have a problem with connecting to Metastore 3.1.
Cheers,
--- gla
On Mon, Dec 9, 2019 at 2:38 PM Priyam Gupta <pr...@zapr.in> wrote:
>
>
> Hi,
>
> In order to have single hive metastore supporting different clusters of
> hive, I have upgraded my hive metastore schema (on postgres) from version
> 1.1.0 to version 2.3.0. Will hive cluster on version 1.1.0 work if I point
> it to upgraded metastore db which is as per hive version 2.3.0.
>
>
> Thanks,
> Priyam
>