You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Alexander Kolbasov <ak...@cloudera.com> on 2018/03/02 18:51:25 UTC
HMS API testing and direct SQL
We currently use two distinct code paths via ObjectStore - the regular one
and direct SQL. So I think all the tests that verify HMS API functionality
should somehow cover both cases.
Peter - what are your thoughts?
- Alex
Re: HMS API testing and direct SQL
Posted by Sergey Shelukhin <se...@hortonworks.com>.
I meant VerifyingObjectStore. I still see it in tests, at least when investigating a random test failure yesterday.
From: Peter Vary <pv...@cloudera.com>>
Date: Tuesday, March 6, 2018 at 08:10
To: Sergey Shelukhin <se...@hortonworks.com>>
Cc: "dev@hive.apache.org<ma...@hive.apache.org>" <de...@hive.apache.org>>
Subject: Re: HMS API testing and direct SQL
Hi Team,
The MetaStore API tests are designed to be easily extensible by new MetaStore configurations. Just take a look at the following class: MetaStoreFactoryForTests. We can add a new configurations here like it is done with the RemoteMetaStore.
What we have to decide is: which is the right level to test DirectSQL methods. MetaStore level or on ObjectStore level. I can see pros and cons about both solutions. What do you think?
Thanks,
Peter
On Mar 2, 2018, at 8:14 PM, Sergey Shelukhin <se...@hortonworks.com>> wrote:
There used to be a special ObjectStore for tests that would run stuff both
with and without DirectSql in tests and fail on mismatch. Not sure about
its state now.
On 18/3/2, 11:05, "Vihang Karajgaonkar" <vi...@cloudera.com>> wrote:
Direct SQL is controlled by configs metastore.try.direct.sql
and metastore.try.direct.sql.ddl which are enabled by default. So I am
guessing the API tests will cover the direct SQL enabled path only unless
the tests override the default configs (I don't think they do).
On Fri, Mar 2, 2018 at 10:51 AM, Alexander Kolbasov <ak...@cloudera.com>>
wrote:
We currently use two distinct code paths via ObjectStore - the regular
one
and direct SQL. So I think all the tests that verify HMS API
functionality
should somehow cover both cases.
Peter - what are your thoughts?
- Alex
Re: HMS API testing and direct SQL
Posted by Alan Gates <al...@gmail.com>.
I would do it at the ObjectStore level. It's faster and will cover 99% of
what we want to test between the two.
Alan.
On Tue, Mar 6, 2018 at 8:10 AM, Peter Vary <pv...@cloudera.com> wrote:
> Hi Team,
>
> The MetaStore API tests are designed to be easily extensible by new
> MetaStore configurations. Just take a look at the following class:
> MetaStoreFactoryForTests. We can add a new configurations here like it is
> done with the RemoteMetaStore.
> What we have to decide is: which is the right level to test DirectSQL
> methods. MetaStore level or on ObjectStore level. I can see pros and cons
> about both solutions. What do you think?
>
> Thanks,
> Peter
>
> > On Mar 2, 2018, at 8:14 PM, Sergey Shelukhin <se...@hortonworks.com>
> wrote:
> >
> > There used to be a special ObjectStore for tests that would run stuff
> both
> > with and without DirectSql in tests and fail on mismatch. Not sure about
> > its state now.
> >
> > On 18/3/2, 11:05, "Vihang Karajgaonkar" <vi...@cloudera.com> wrote:
> >
> >> Direct SQL is controlled by configs metastore.try.direct.sql
> >> and metastore.try.direct.sql.ddl which are enabled by default. So I am
> >> guessing the API tests will cover the direct SQL enabled path only
> unless
> >> the tests override the default configs (I don't think they do).
> >>
> >> On Fri, Mar 2, 2018 at 10:51 AM, Alexander Kolbasov <akolb@cloudera.com
> >
> >> wrote:
> >>
> >>> We currently use two distinct code paths via ObjectStore - the regular
> >>> one
> >>> and direct SQL. So I think all the tests that verify HMS API
> >>> functionality
> >>> should somehow cover both cases.
> >>>
> >>> Peter - what are your thoughts?
> >>>
> >>> - Alex
> >>>
> >
>
>
Re: HMS API testing and direct SQL
Posted by Peter Vary <pv...@cloudera.com>.
Hi Team,
The MetaStore API tests are designed to be easily extensible by new MetaStore configurations. Just take a look at the following class: MetaStoreFactoryForTests. We can add a new configurations here like it is done with the RemoteMetaStore.
What we have to decide is: which is the right level to test DirectSQL methods. MetaStore level or on ObjectStore level. I can see pros and cons about both solutions. What do you think?
Thanks,
Peter
> On Mar 2, 2018, at 8:14 PM, Sergey Shelukhin <se...@hortonworks.com> wrote:
>
> There used to be a special ObjectStore for tests that would run stuff both
> with and without DirectSql in tests and fail on mismatch. Not sure about
> its state now.
>
> On 18/3/2, 11:05, "Vihang Karajgaonkar" <vi...@cloudera.com> wrote:
>
>> Direct SQL is controlled by configs metastore.try.direct.sql
>> and metastore.try.direct.sql.ddl which are enabled by default. So I am
>> guessing the API tests will cover the direct SQL enabled path only unless
>> the tests override the default configs (I don't think they do).
>>
>> On Fri, Mar 2, 2018 at 10:51 AM, Alexander Kolbasov <ak...@cloudera.com>
>> wrote:
>>
>>> We currently use two distinct code paths via ObjectStore - the regular
>>> one
>>> and direct SQL. So I think all the tests that verify HMS API
>>> functionality
>>> should somehow cover both cases.
>>>
>>> Peter - what are your thoughts?
>>>
>>> - Alex
>>>
>
Re: HMS API testing and direct SQL
Posted by Sergey Shelukhin <se...@hortonworks.com>.
There used to be a special ObjectStore for tests that would run stuff both
with and without DirectSql in tests and fail on mismatch. Not sure about
its state now.
On 18/3/2, 11:05, "Vihang Karajgaonkar" <vi...@cloudera.com> wrote:
>Direct SQL is controlled by configs metastore.try.direct.sql
>and metastore.try.direct.sql.ddl which are enabled by default. So I am
>guessing the API tests will cover the direct SQL enabled path only unless
>the tests override the default configs (I don't think they do).
>
>On Fri, Mar 2, 2018 at 10:51 AM, Alexander Kolbasov <ak...@cloudera.com>
>wrote:
>
>> We currently use two distinct code paths via ObjectStore - the regular
>>one
>> and direct SQL. So I think all the tests that verify HMS API
>>functionality
>> should somehow cover both cases.
>>
>> Peter - what are your thoughts?
>>
>> - Alex
>>
Re: HMS API testing and direct SQL
Posted by Vihang Karajgaonkar <vi...@cloudera.com>.
Direct SQL is controlled by configs metastore.try.direct.sql
and metastore.try.direct.sql.ddl which are enabled by default. So I am
guessing the API tests will cover the direct SQL enabled path only unless
the tests override the default configs (I don't think they do).
On Fri, Mar 2, 2018 at 10:51 AM, Alexander Kolbasov <ak...@cloudera.com>
wrote:
> We currently use two distinct code paths via ObjectStore - the regular one
> and direct SQL. So I think all the tests that verify HMS API functionality
> should somehow cover both cases.
>
> Peter - what are your thoughts?
>
> - Alex
>