You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Sergio Pena <se...@cloudera.com> on 2015/03/24 00:17:09 UTC
New HMS Upgrade tests on Jenkins
Hi All,
I configured a new jenkins job that will run HMS upgrade tests whenever a
change on metastore/scripts/upgrade/ files is detected.
You may receive an extra test comment from Hive QA when these tests pass or
fail only from patches that make the change.
Currently, the HMS upgrade tests run on MySQL; but we're working on cover
the rest of the DBs.
- Sergio
Re: New HMS Upgrade tests on Jenkins
Posted by Sergio Pena <se...@cloudera.com>.
This sounds great. But the current HMS upgrade scripts tests cannot be used
for the precommit testing. These scripts use LXC to prepare different DB
environments in a single Amazon instance. Because HMS upgrade tests are run
quickly, then we can reuse the same instance to run different DB servers
(one per LXC container).
However, I like this idea of running the precommit tests on Mysql. If we
take advantage of the concurrency, then we might get tests results faster.
I'll take a look at the code, and see how we can improve it to use Mysql.
On Thu, Apr 9, 2015 at 1:40 PM, Szehon Ho <sz...@cloudera.com> wrote:
> Good idea guys. Would leave to Sergio to comment on feasibility from
> infrastructure perspective.
>
> But first maybe someone else can volunteer to make the Qtest framework able
> to take arguments about db type, jdbc url, etc and make sure it works
> locally. Then we can integrate into the Precommit tests. I think existing
> QtestUtil environment variable arguments mechanism would work, keeping the
> current integration with Precommit tests.
>
> Thanks
> Szehon
>
>
>
> On Thu, Apr 9, 2015 at 10:38 AM, Eugene Koifman <ek...@hortonworks.com>
> wrote:
>
> > Using mysql would also also concurrency tests with commands from multiple
> > clients
> > which you can¹t do with derby. This would be a big plus.
> >
> > On 4/8/15, 11:00 AM, "Ashutosh Chauhan" <ha...@apache.org> wrote:
> >
> > >I wonder if this infra (or parts of it) can be used to run our pre
> commit
> > >tests on mysql. Currently, it runs on derby which is never a production
> > >choice. So, many bugs go in trunk unnoticed, until its too late. If we
> can
> > >run our tests using mysql backend that will be of much more value.
> > >
> > >Thoughts?
> > >
> > >-Ashutosh
> > >
> > >On Mon, Mar 23, 2015 at 4:38 PM, Ashutosh Chauhan <hashutosh@apache.org
> >
> > >wrote:
> > >
> > >> Awesome work, Sergio!
> > >> Much appreciated.
> > >>
> > >> Thanks for doing this,
> > >> Ashutosh
> > >>
> > >> On Mon, Mar 23, 2015 at 4:17 PM, Sergio Pena <
> sergio.pena@cloudera.com>
> > >> wrote:
> > >>
> > >>> Hi All,
> > >>>
> > >>> I configured a new jenkins job that will run HMS upgrade tests
> > >>>whenever a
> > >>> change on metastore/scripts/upgrade/ files is detected.
> > >>>
> > >>> You may receive an extra test comment from Hive QA when these tests
> > >>>pass
> > >>> or
> > >>> fail only from patches that make the change.
> > >>>
> > >>> Currently, the HMS upgrade tests run on MySQL; but we're working on
> > >>>cover
> > >>> the rest of the DBs.
> > >>>
> > >>> - Sergio
> > >>>
> > >>
> > >>
> >
> >
>
Re: New HMS Upgrade tests on Jenkins
Posted by Szehon Ho <sz...@cloudera.com>.
Good idea guys. Would leave to Sergio to comment on feasibility from
infrastructure perspective.
But first maybe someone else can volunteer to make the Qtest framework able
to take arguments about db type, jdbc url, etc and make sure it works
locally. Then we can integrate into the Precommit tests. I think existing
QtestUtil environment variable arguments mechanism would work, keeping the
current integration with Precommit tests.
Thanks
Szehon
On Thu, Apr 9, 2015 at 10:38 AM, Eugene Koifman <ek...@hortonworks.com>
wrote:
> Using mysql would also also concurrency tests with commands from multiple
> clients
> which you can¹t do with derby. This would be a big plus.
>
> On 4/8/15, 11:00 AM, "Ashutosh Chauhan" <ha...@apache.org> wrote:
>
> >I wonder if this infra (or parts of it) can be used to run our pre commit
> >tests on mysql. Currently, it runs on derby which is never a production
> >choice. So, many bugs go in trunk unnoticed, until its too late. If we can
> >run our tests using mysql backend that will be of much more value.
> >
> >Thoughts?
> >
> >-Ashutosh
> >
> >On Mon, Mar 23, 2015 at 4:38 PM, Ashutosh Chauhan <ha...@apache.org>
> >wrote:
> >
> >> Awesome work, Sergio!
> >> Much appreciated.
> >>
> >> Thanks for doing this,
> >> Ashutosh
> >>
> >> On Mon, Mar 23, 2015 at 4:17 PM, Sergio Pena <se...@cloudera.com>
> >> wrote:
> >>
> >>> Hi All,
> >>>
> >>> I configured a new jenkins job that will run HMS upgrade tests
> >>>whenever a
> >>> change on metastore/scripts/upgrade/ files is detected.
> >>>
> >>> You may receive an extra test comment from Hive QA when these tests
> >>>pass
> >>> or
> >>> fail only from patches that make the change.
> >>>
> >>> Currently, the HMS upgrade tests run on MySQL; but we're working on
> >>>cover
> >>> the rest of the DBs.
> >>>
> >>> - Sergio
> >>>
> >>
> >>
>
>
Re: New HMS Upgrade tests on Jenkins
Posted by Eugene Koifman <ek...@hortonworks.com>.
Using mysql would also also concurrency tests with commands from multiple
clients
which you can¹t do with derby. This would be a big plus.
On 4/8/15, 11:00 AM, "Ashutosh Chauhan" <ha...@apache.org> wrote:
>I wonder if this infra (or parts of it) can be used to run our pre commit
>tests on mysql. Currently, it runs on derby which is never a production
>choice. So, many bugs go in trunk unnoticed, until its too late. If we can
>run our tests using mysql backend that will be of much more value.
>
>Thoughts?
>
>-Ashutosh
>
>On Mon, Mar 23, 2015 at 4:38 PM, Ashutosh Chauhan <ha...@apache.org>
>wrote:
>
>> Awesome work, Sergio!
>> Much appreciated.
>>
>> Thanks for doing this,
>> Ashutosh
>>
>> On Mon, Mar 23, 2015 at 4:17 PM, Sergio Pena <se...@cloudera.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> I configured a new jenkins job that will run HMS upgrade tests
>>>whenever a
>>> change on metastore/scripts/upgrade/ files is detected.
>>>
>>> You may receive an extra test comment from Hive QA when these tests
>>>pass
>>> or
>>> fail only from patches that make the change.
>>>
>>> Currently, the HMS upgrade tests run on MySQL; but we're working on
>>>cover
>>> the rest of the DBs.
>>>
>>> - Sergio
>>>
>>
>>
Re: New HMS Upgrade tests on Jenkins
Posted by Ashutosh Chauhan <ha...@apache.org>.
I wonder if this infra (or parts of it) can be used to run our pre commit
tests on mysql. Currently, it runs on derby which is never a production
choice. So, many bugs go in trunk unnoticed, until its too late. If we can
run our tests using mysql backend that will be of much more value.
Thoughts?
-Ashutosh
On Mon, Mar 23, 2015 at 4:38 PM, Ashutosh Chauhan <ha...@apache.org>
wrote:
> Awesome work, Sergio!
> Much appreciated.
>
> Thanks for doing this,
> Ashutosh
>
> On Mon, Mar 23, 2015 at 4:17 PM, Sergio Pena <se...@cloudera.com>
> wrote:
>
>> Hi All,
>>
>> I configured a new jenkins job that will run HMS upgrade tests whenever a
>> change on metastore/scripts/upgrade/ files is detected.
>>
>> You may receive an extra test comment from Hive QA when these tests pass
>> or
>> fail only from patches that make the change.
>>
>> Currently, the HMS upgrade tests run on MySQL; but we're working on cover
>> the rest of the DBs.
>>
>> - Sergio
>>
>
>
Re: New HMS Upgrade tests on Jenkins
Posted by Ashutosh Chauhan <ha...@apache.org>.
Awesome work, Sergio!
Much appreciated.
Thanks for doing this,
Ashutosh
On Mon, Mar 23, 2015 at 4:17 PM, Sergio Pena <se...@cloudera.com>
wrote:
> Hi All,
>
> I configured a new jenkins job that will run HMS upgrade tests whenever a
> change on metastore/scripts/upgrade/ files is detected.
>
> You may receive an extra test comment from Hive QA when these tests pass or
> fail only from patches that make the change.
>
> Currently, the HMS upgrade tests run on MySQL; but we're working on cover
> the rest of the DBs.
>
> - Sergio
>