Long running time for recently added tests in standalone-metastore

[Alan Gates <al...@gmail.com>]

With all of the added tests in standalone-metastore/…/client directory, the
runtime of ‘mvn test’ in standalone-metastore went from 6 minutes to 26 on
my humble laptop.  We do not want to get ourselves back where the rest of
Hive is; currently Hive developers don’t run the unit tests themselves
because the tests take too long.  I believe we should be working to push
the unit test runtime down to about 2 minutes, so people are willing to run
it frequently as part of their development.

I don’t mean that the new tests aren't valuable.  But we need a balance
between test coverage in the unit tests and usability.  So I propose that
we carve off many of the current unit tests (including some not in the
client module, like TestSetUGI…, TestRemote...) in a new profile,
‘nightly’, or ‘checkin’, or something.  Then ‘mvn install’ will finish
quickly while hopefully covering 90% of the areas we need to cover.  We can
ask developers to run the extended set before checkin and configure the
automated tests to do the same.  This way we still cover everything before
committing.

Seem reasonable?

Alan.

Re: Long running time for recently added tests in standalone-metastore

[Vihang Karajgaonkar <vi...@cloudera.com>]

+1 I think thats a great idea. Hopefully, this will help reducing some load
on HiveQA jobs as well. Currently, most of the patches go through
multi-iteration HiveQA runs at-least some of which could have been avoided
if there is a light-weight test profile which can be run locally. This
would be a win-win for both the devs and the faster turn-around from HiveQA
jobs. We should probably think about extending this idea to Hive as well as
a separate effort to improve the efficiency of HiveQA runs.

On Tue, Jan 30, 2018 at 1:19 AM, Peter Vary <pv...@cloudera.com> wrote:

> +1
>
> > On Jan 30, 2018, at 1:54 AM, Owen O'Malley <ow...@gmail.com>
> wrote:
> >
> > +1
> >
> > On Mon, Jan 29, 2018 at 4:38 PM, Alexander Kolbasov <ak...@cloudera.com>
> > wrote:
> >
> >> Quite reasonable.
> >> +1
> >>
> >> On Mon, Jan 29, 2018 at 4:19 PM, Alan Gates <al...@gmail.com>
> wrote:
> >>
> >>> With all of the added tests in standalone-metastore/…/client directory,
> >> the
> >>> runtime of ‘mvn test’ in standalone-metastore went from 6 minutes to 26
> >> on
> >>> my humble laptop.  We do not want to get ourselves back where the rest
> of
> >>> Hive is; currently Hive developers don’t run the unit tests themselves
> >>> because the tests take too long.  I believe we should be working to
> push
> >>> the unit test runtime down to about 2 minutes, so people are willing to
> >> run
> >>> it frequently as part of their development.
> >>>
> >>> I don’t mean that the new tests aren't valuable.  But we need a balance
> >>> between test coverage in the unit tests and usability.  So I propose
> that
> >>> we carve off many of the current unit tests (including some not in the
> >>> client module, like TestSetUGI…, TestRemote...) in a new profile,
> >>> ‘nightly’, or ‘checkin’, or something.  Then ‘mvn install’ will finish
> >>> quickly while hopefully covering 90% of the areas we need to cover.  We
> >> can
> >>> ask developers to run the extended set before checkin and configure the
> >>> automated tests to do the same.  This way we still cover everything
> >> before
> >>> committing.
> >>>
> >>> Seem reasonable?
> >>>
> >>> Alan.
> >>>
> >>
>
>

Re: Long running time for recently added tests in standalone-metastore

[Peter Vary <pv...@cloudera.com>]

+1

> On Jan 30, 2018, at 1:54 AM, Owen O'Malley <ow...@gmail.com> wrote:
> 
> +1
> 
> On Mon, Jan 29, 2018 at 4:38 PM, Alexander Kolbasov <ak...@cloudera.com>
> wrote:
> 
>> Quite reasonable.
>> +1
>> 
>> On Mon, Jan 29, 2018 at 4:19 PM, Alan Gates <al...@gmail.com> wrote:
>> 
>>> With all of the added tests in standalone-metastore/…/client directory,
>> the
>>> runtime of ‘mvn test’ in standalone-metastore went from 6 minutes to 26
>> on
>>> my humble laptop.  We do not want to get ourselves back where the rest of
>>> Hive is; currently Hive developers don’t run the unit tests themselves
>>> because the tests take too long.  I believe we should be working to push
>>> the unit test runtime down to about 2 minutes, so people are willing to
>> run
>>> it frequently as part of their development.
>>> 
>>> I don’t mean that the new tests aren't valuable.  But we need a balance
>>> between test coverage in the unit tests and usability.  So I propose that
>>> we carve off many of the current unit tests (including some not in the
>>> client module, like TestSetUGI…, TestRemote...) in a new profile,
>>> ‘nightly’, or ‘checkin’, or something.  Then ‘mvn install’ will finish
>>> quickly while hopefully covering 90% of the areas we need to cover.  We
>> can
>>> ask developers to run the extended set before checkin and configure the
>>> automated tests to do the same.  This way we still cover everything
>> before
>>> committing.
>>> 
>>> Seem reasonable?
>>> 
>>> Alan.
>>> 
>> 

Re: Long running time for recently added tests in standalone-metastore

[Owen O'Malley <ow...@gmail.com>]

+1

On Mon, Jan 29, 2018 at 4:38 PM, Alexander Kolbasov <ak...@cloudera.com>
wrote:

> Quite reasonable.
> +1
>
> On Mon, Jan 29, 2018 at 4:19 PM, Alan Gates <al...@gmail.com> wrote:
>
> > With all of the added tests in standalone-metastore/…/client directory,
> the
> > runtime of ‘mvn test’ in standalone-metastore went from 6 minutes to 26
> on
> > my humble laptop.  We do not want to get ourselves back where the rest of
> > Hive is; currently Hive developers don’t run the unit tests themselves
> > because the tests take too long.  I believe we should be working to push
> > the unit test runtime down to about 2 minutes, so people are willing to
> run
> > it frequently as part of their development.
> >
> > I don’t mean that the new tests aren't valuable.  But we need a balance
> > between test coverage in the unit tests and usability.  So I propose that
> > we carve off many of the current unit tests (including some not in the
> > client module, like TestSetUGI…, TestRemote...) in a new profile,
> > ‘nightly’, or ‘checkin’, or something.  Then ‘mvn install’ will finish
> > quickly while hopefully covering 90% of the areas we need to cover.  We
> can
> > ask developers to run the extended set before checkin and configure the
> > automated tests to do the same.  This way we still cover everything
> before
> > committing.
> >
> > Seem reasonable?
> >
> > Alan.
> >
>

Re: Long running time for recently added tests in standalone-metastore

[Alexander Kolbasov <ak...@cloudera.com>]

Quite reasonable.
+1

On Mon, Jan 29, 2018 at 4:19 PM, Alan Gates <al...@gmail.com> wrote:

> With all of the added tests in standalone-metastore/…/client directory, the
> runtime of ‘mvn test’ in standalone-metastore went from 6 minutes to 26 on
> my humble laptop.  We do not want to get ourselves back where the rest of
> Hive is; currently Hive developers don’t run the unit tests themselves
> because the tests take too long.  I believe we should be working to push
> the unit test runtime down to about 2 minutes, so people are willing to run
> it frequently as part of their development.
>
> I don’t mean that the new tests aren't valuable.  But we need a balance
> between test coverage in the unit tests and usability.  So I propose that
> we carve off many of the current unit tests (including some not in the
> client module, like TestSetUGI…, TestRemote...) in a new profile,
> ‘nightly’, or ‘checkin’, or something.  Then ‘mvn install’ will finish
> quickly while hopefully covering 90% of the areas we need to cover.  We can
> ask developers to run the extended set before checkin and configure the
> automated tests to do the same.  This way we still cover everything before
> committing.
>
> Seem reasonable?
>
> Alan.
>