[Discuss] MariaDB support

[Thejas Nair <th...@gmail.com>]

There seems to be increasing interest in supporting MariaDB as an
option for storing metastore metadata. Supporting it as a database
option is also easy as it is compatible with mysql. I thought it would
be useful to discuss supporting it in the dev list before creating any
jiras.

There are two aspects I would like to discuss -

1. Changes in hive to support MariaDB

The existing mysql schema creation/upgrade scripts in hive should just
work for mariadb as well.
However, MariaDB has some additional optimizations that we might want
to use in future to optimize queries for it. That would mean creating
specific scripts for mariadb.

However, until we introduce such MariaDB specific tuning, I think it
is better to avoid duplicating the mysql scripts.

To make the transition to possibly using MariaDB optimized scripts
easier, one option is to have schematool consider it as an alias for
mysql until that happens.


2. Testing with MariaDB
It would be useful to have tests for mariadb as well on the lines of
what is available for mysql in
https://issues.apache.org/jira/browse/HIVE-9800, to ensure that
mariadb support is not broken.

Thanks,
Thejas

Re: [Discuss] MariaDB support

[Sergio Pena <se...@cloudera.com>]

I like the idea.
+1 to both points.

I think we can use a symlink on the upgrade scripts for 'mariadb -> mysql'
so that we re-use all .sql scripts for Maridb.
For our upgrade tests, we still need to create a prepare.sh/executed.sh
script that installs mariadb and executes the mariadb command to run the
tests.

On Wed, Mar 16, 2016 at 4:42 PM, Sushanth Sowmyan <kh...@gmail.com>
wrote:

> +1 to introduction of mariadb support - I think it's important that we
> support MariaDB - there is an increasing interest in the broader open
> source community of migrating from mysql to either postgres or
> mariadb. While they're compatible now, it's important that we be aware
> of gotchas that come up, which we'll be aware of only after there is
> active usage.
>
> +1 to not duplicating mysql scripts unless we find a need to diverge,
> and having schematool consider it an alias for now.
>
> On Wed, Mar 16, 2016 at 12:09 PM, Szehon Ho <sz...@cloudera.com> wrote:
> > Yea, +1 to point 2.
> >
> > For point one, I also agree that it is compatible with mysql and not be a
> > ton of work unless you want to optimize, on our observations we have seen
> > existing mysql scripts work fine against mariadb.
> >
> > On Wed, Mar 16, 2016 at 12:04 PM, Dmitry Tolpeko <dm...@gmail.com>
> > wrote:
> >>
> >> +1 great idea
> >>
> >> On Wed, Mar 16, 2016 at 10:00 PM, Thejas Nair <th...@gmail.com>
> >> wrote:
> >>>
> >>> + Sergio, Szehon, Ashutosh, Sushanth, Sergey,
> >>>
> >>> Any thoughts on this ?
> >>>
> >>>
> >>> On Tue, Mar 15, 2016 at 7:08 PM, Thejas Nair <th...@gmail.com>
> >>> wrote:
> >>> > There seems to be increasing interest in supporting MariaDB as an
> >>> > option for storing metastore metadata. Supporting it as a database
> >>> > option is also easy as it is compatible with mysql. I thought it
> would
> >>> > be useful to discuss supporting it in the dev list before creating
> any
> >>> > jiras.
> >>> >
> >>> > There are two aspects I would like to discuss -
> >>> >
> >>> > 1. Changes in hive to support MariaDB
> >>> >
> >>> > The existing mysql schema creation/upgrade scripts in hive should
> just
> >>> > work for mariadb as well.
> >>> > However, MariaDB has some additional optimizations that we might want
> >>> > to use in future to optimize queries for it. That would mean creating
> >>> > specific scripts for mariadb.
> >>> >
> >>> > However, until we introduce such MariaDB specific tuning, I think it
> >>> > is better to avoid duplicating the mysql scripts.
> >>> >
> >>> > To make the transition to possibly using MariaDB optimized scripts
> >>> > easier, one option is to have schematool consider it as an alias for
> >>> > mysql until that happens.
> >>> >
> >>> >
> >>> > 2. Testing with MariaDB
> >>> > It would be useful to have tests for mariadb as well on the lines of
> >>> > what is available for mysql in
> >>> > https://issues.apache.org/jira/browse/HIVE-9800, to ensure that
> >>> > mariadb support is not broken.
> >>> >
> >>> > Thanks,
> >>> > Thejas
> >>
> >>
> >
>

Re: [Discuss] MariaDB support

[Sushanth Sowmyan <kh...@gmail.com>]

+1 to introduction of mariadb support - I think it's important that we
support MariaDB - there is an increasing interest in the broader open
source community of migrating from mysql to either postgres or
mariadb. While they're compatible now, it's important that we be aware
of gotchas that come up, which we'll be aware of only after there is
active usage.

+1 to not duplicating mysql scripts unless we find a need to diverge,
and having schematool consider it an alias for now.

On Wed, Mar 16, 2016 at 12:09 PM, Szehon Ho <sz...@cloudera.com> wrote:
> Yea, +1 to point 2.
>
> For point one, I also agree that it is compatible with mysql and not be a
> ton of work unless you want to optimize, on our observations we have seen
> existing mysql scripts work fine against mariadb.
>
> On Wed, Mar 16, 2016 at 12:04 PM, Dmitry Tolpeko <dm...@gmail.com>
> wrote:
>>
>> +1 great idea
>>
>> On Wed, Mar 16, 2016 at 10:00 PM, Thejas Nair <th...@gmail.com>
>> wrote:
>>>
>>> + Sergio, Szehon, Ashutosh, Sushanth, Sergey,
>>>
>>> Any thoughts on this ?
>>>
>>>
>>> On Tue, Mar 15, 2016 at 7:08 PM, Thejas Nair <th...@gmail.com>
>>> wrote:
>>> > There seems to be increasing interest in supporting MariaDB as an
>>> > option for storing metastore metadata. Supporting it as a database
>>> > option is also easy as it is compatible with mysql. I thought it would
>>> > be useful to discuss supporting it in the dev list before creating any
>>> > jiras.
>>> >
>>> > There are two aspects I would like to discuss -
>>> >
>>> > 1. Changes in hive to support MariaDB
>>> >
>>> > The existing mysql schema creation/upgrade scripts in hive should just
>>> > work for mariadb as well.
>>> > However, MariaDB has some additional optimizations that we might want
>>> > to use in future to optimize queries for it. That would mean creating
>>> > specific scripts for mariadb.
>>> >
>>> > However, until we introduce such MariaDB specific tuning, I think it
>>> > is better to avoid duplicating the mysql scripts.
>>> >
>>> > To make the transition to possibly using MariaDB optimized scripts
>>> > easier, one option is to have schematool consider it as an alias for
>>> > mysql until that happens.
>>> >
>>> >
>>> > 2. Testing with MariaDB
>>> > It would be useful to have tests for mariadb as well on the lines of
>>> > what is available for mysql in
>>> > https://issues.apache.org/jira/browse/HIVE-9800, to ensure that
>>> > mariadb support is not broken.
>>> >
>>> > Thanks,
>>> > Thejas
>>
>>
>

Re: [Discuss] MariaDB support

[Szehon Ho <sz...@cloudera.com>]

Yea, +1 to point 2.

For point one, I also agree that it is compatible with mysql and not be a
ton of work unless you want to optimize, on our observations we have seen
existing mysql scripts work fine against mariadb.

On Wed, Mar 16, 2016 at 12:04 PM, Dmitry Tolpeko <dm...@gmail.com>
wrote:

> +1 great idea
>
> On Wed, Mar 16, 2016 at 10:00 PM, Thejas Nair <th...@gmail.com>
> wrote:
>
>> + Sergio, Szehon, Ashutosh, Sushanth, Sergey,
>>
>> Any thoughts on this ?
>>
>>
>> On Tue, Mar 15, 2016 at 7:08 PM, Thejas Nair <th...@gmail.com>
>> wrote:
>> > There seems to be increasing interest in supporting MariaDB as an
>> > option for storing metastore metadata. Supporting it as a database
>> > option is also easy as it is compatible with mysql. I thought it would
>> > be useful to discuss supporting it in the dev list before creating any
>> > jiras.
>> >
>> > There are two aspects I would like to discuss -
>> >
>> > 1. Changes in hive to support MariaDB
>> >
>> > The existing mysql schema creation/upgrade scripts in hive should just
>> > work for mariadb as well.
>> > However, MariaDB has some additional optimizations that we might want
>> > to use in future to optimize queries for it. That would mean creating
>> > specific scripts for mariadb.
>> >
>> > However, until we introduce such MariaDB specific tuning, I think it
>> > is better to avoid duplicating the mysql scripts.
>> >
>> > To make the transition to possibly using MariaDB optimized scripts
>> > easier, one option is to have schematool consider it as an alias for
>> > mysql until that happens.
>> >
>> >
>> > 2. Testing with MariaDB
>> > It would be useful to have tests for mariadb as well on the lines of
>> > what is available for mysql in
>> > https://issues.apache.org/jira/browse/HIVE-9800, to ensure that
>> > mariadb support is not broken.
>> >
>> > Thanks,
>> > Thejas
>>
>
>

Re: [Discuss] MariaDB support

[Dmitry Tolpeko <dm...@gmail.com>]

+1 great idea

On Wed, Mar 16, 2016 at 10:00 PM, Thejas Nair <th...@gmail.com> wrote:

> + Sergio, Szehon, Ashutosh, Sushanth, Sergey,
>
> Any thoughts on this ?
>
>
> On Tue, Mar 15, 2016 at 7:08 PM, Thejas Nair <th...@gmail.com>
> wrote:
> > There seems to be increasing interest in supporting MariaDB as an
> > option for storing metastore metadata. Supporting it as a database
> > option is also easy as it is compatible with mysql. I thought it would
> > be useful to discuss supporting it in the dev list before creating any
> > jiras.
> >
> > There are two aspects I would like to discuss -
> >
> > 1. Changes in hive to support MariaDB
> >
> > The existing mysql schema creation/upgrade scripts in hive should just
> > work for mariadb as well.
> > However, MariaDB has some additional optimizations that we might want
> > to use in future to optimize queries for it. That would mean creating
> > specific scripts for mariadb.
> >
> > However, until we introduce such MariaDB specific tuning, I think it
> > is better to avoid duplicating the mysql scripts.
> >
> > To make the transition to possibly using MariaDB optimized scripts
> > easier, one option is to have schematool consider it as an alias for
> > mysql until that happens.
> >
> >
> > 2. Testing with MariaDB
> > It would be useful to have tests for mariadb as well on the lines of
> > what is available for mysql in
> > https://issues.apache.org/jira/browse/HIVE-9800, to ensure that
> > mariadb support is not broken.
> >
> > Thanks,
> > Thejas
>

Re: [Discuss] MariaDB support

[Thejas Nair <th...@gmail.com>]

+ Sergio, Szehon, Ashutosh, Sushanth, Sergey,

Any thoughts on this ?


On Tue, Mar 15, 2016 at 7:08 PM, Thejas Nair <th...@gmail.com> wrote:
> There seems to be increasing interest in supporting MariaDB as an
> option for storing metastore metadata. Supporting it as a database
> option is also easy as it is compatible with mysql. I thought it would
> be useful to discuss supporting it in the dev list before creating any
> jiras.
>
> There are two aspects I would like to discuss -
>
> 1. Changes in hive to support MariaDB
>
> The existing mysql schema creation/upgrade scripts in hive should just
> work for mariadb as well.
> However, MariaDB has some additional optimizations that we might want
> to use in future to optimize queries for it. That would mean creating
> specific scripts for mariadb.
>
> However, until we introduce such MariaDB specific tuning, I think it
> is better to avoid duplicating the mysql scripts.
>
> To make the transition to possibly using MariaDB optimized scripts
> easier, one option is to have schematool consider it as an alias for
> mysql until that happens.
>
>
> 2. Testing with MariaDB
> It would be useful to have tests for mariadb as well on the lines of
> what is available for mysql in
> https://issues.apache.org/jira/browse/HIVE-9800, to ensure that
> mariadb support is not broken.
>
> Thanks,
> Thejas