Hive Metastore on Amazon Aurora

[Elliot West <te...@gmail.com>]

Hello,

Is anyone running the Hive metastore database on Amazon Aurora?:
https://aws.amazon.com/rds/aurora/details/. My expectation is that it
should work nicely as it is derived from MySQL but I'd be keen to hear of
user's experiences with this setup.

Many thanks,

Elliot.

Re: Hive Metastore on Amazon Aurora

[Elliot West <te...@gmail.com>]

Hi Mich,

Correct. We have proof of concepts up and running with both MySQL on RDS
and Aurora. We'd be keen to hear of experiences of others with Aurora in a
Hive metastore database role, primarily as a sanity check. In answer to
your specific points:

   1. 30GB
   2. We don't intend to use ACID in this scenario (currently).

For this application we particularly value:

   - Compatibility (with Hive)
   - High availability
   - Scalability
   - Ease of management

Thanks, Elliot.

On 11 July 2016 at 15:15, Mich Talebzadeh <mi...@gmail.com> wrote:

> Hi  Elliot,
>
> Am I correct that you want to put your Hive metastore on Amazon? Is the
> metastore (database/schema) is sitting on on MySQL and you want to migrate
> your MySQL to cloud now?
>
> Two questions that need to be verified
>
>
>    1. How big is your current metadata
>    2. Do you do a lot of transaction activity using ORC files with
>    Insert/Update/Delete that need to communicate with metastore with heartbeat
>    etc?
>
>
> HTH
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 11 July 2016 at 13:58, Elliot West <te...@gmail.com> wrote:
>
>> Hello,
>>
>> Is anyone running the Hive metastore database on Amazon Aurora?:
>> https://aws.amazon.com/rds/aurora/details/. My expectation is that it
>> should work nicely as it is derived from MySQL but I'd be keen to hear of
>> user's experiences with this setup.
>>
>> Many thanks,
>>
>> Elliot.
>>
>
>

Re: Hive Metastore on Amazon Aurora

[Mich Talebzadeh <mi...@gmail.com>]

Hi  Elliot,

Am I correct that you want to put your Hive metastore on Amazon? Is the
metastore (database/schema) is sitting on on MySQL and you want to migrate
your MySQL to cloud now?

Two questions that need to be verified


   1. How big is your current metadata
   2. Do you do a lot of transaction activity using ORC files with
   Insert/Update/Delete that need to communicate with metastore with heartbeat
   etc?


HTH


Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.



On 11 July 2016 at 13:58, Elliot West <te...@gmail.com> wrote:

> Hello,
>
> Is anyone running the Hive metastore database on Amazon Aurora?:
> https://aws.amazon.com/rds/aurora/details/. My expectation is that it
> should work nicely as it is derived from MySQL but I'd be keen to hear of
> user's experiences with this setup.
>
> Many thanks,
>
> Elliot.
>