You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Stack <st...@duboce.net> on 2015/01/01 01:22:20 UTC
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
I upped hadoopqa retention to keep last 100 builds and or last 7 days,
whichever comes first.
St.Ack
On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net> wrote:
> Branch-1 and master have stabilized and now run mostly blue (give or take
> the odd failure) [1][2]. Having a mostly blue branch-1 has helped us
> identify at least one destabilizing commit in the last few days, maybe two;
> this is as it should be (smile).
>
> Lets keep our builds blue. If you commit a patch, make sure subsequent
> builds stay blue. You can subscribe to builds@hbase.apache.org to get
> notice of failures if not already subscribed.
>
> Thanks,
> St.Ack
>
> 1. https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> 2. https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>
>
> On Mon, Oct 13, 2014 at 4:41 PM, Stack <st...@duboce.net> wrote:
>
>> A few notes on testing.
>>
>> Too long to read, infra is more capable now and after some work, we are
>> seeing branch-1 and trunk mostly running blue. Lets try and keep it this
>> way going forward.
>>
>> Apache Infra has new, more capable hardware.
>>
>> A recent spurt of test fixing combined with more capable hardware seems
>> to have gotten us to a new place; tests are mostly passing now on branch-1
>> and master. Lets try and keep it this way and start to trust our test runs
>> again. Just a few flakies remain. Lets try and nail them.
>>
>> Our tests now run in parallel with other test suites where previous we
>> ran alone. You can see this sometimes when our zombie detector reports
>> tests from another project altogether as lingerers (To be fixed). Some of
>> our tests are failing because a concurrent hbase run is undoing classes and
>> data from under it. Also, lets fix.
>>
>> Our tests are brittle. It takes 75minutes for them to complete. Many are
>> heavy-duty integration tests starting up multiple clusters and mapreduce
>> all in the one JVM. It is a miracle they pass at all. Usually integration
>> tests have been cast as unit tests because there was no where else for them
>> to get an airing. We have the hbase-it suite now which would be a more apt
>> place but until these are run on a regular basis in public for all to see,
>> the fat integration tests disguised as unit tests will remain. A review of
>> our current unit tests weeding the old cruft and the no longer relevant or
>> duplicates would be a nice undertaking if someone is looking to contribute.
>>
>> Alex Newman has been working on making our tests work up on travis and
>> circle-ci. That'll be sweet when it goes end-to-end. He also added in
>> some "type" categorizations -- client, filter, mapreduce -- alongside our
>> old "sizing" categorizations of small/medium/large. His thinking is that
>> we can run these categorizations in parallel so we could run the total
>> suite in about the time of the longest test, say 20-30minutes? We could
>> even change Apache to run them this way.
>>
>> FYI,
>> St.Ack
>>
>>
>>
>>
>>
>>
>>
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
updated the precommit patch testing job to remove the failed-build-specific
description pattern (it was set to match HDFS jira ids) so that everything
will use the hbase jira matching regex.
On Mon, Jun 22, 2015 at 11:48 PM, Sean Busbey <bu...@cloudera.com> wrote:
> I moved TestBulkIngest and TestIngest out to HBASE-13952 and HBASE-13953,
> respectively.
>
> added versions of the remaining tests that runs on branch-1 and master.
> presuming things come back passing I'll turn on notifications tomorrow.
>
> On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> I've add a new build test to run the ITs under sunny day conditions using
>> minicluster for both java 7 and java 8 on the 1.2 line.
>>
>> https://builds.apache.org/job/HBase-1.2-IT/
>>
>> I haven't turned on notifications yet, because BulkIngest and TestIngest
>> are read. details on HBASE-13750
>>
>> On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com> wrote:
>>
>>> sigh. that should have been "is now a matrix build".
>>>
>>> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com>
>>> wrote:
>>>
>>>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 in
>>>> parallel.
>>>>
>>>> I saved the old job for now and left it disabled[2].
>>>>
>>>>
>>>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
>>>> [2]:
>>>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>>>>
>>>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com>
>>>> wrote:
>>>>
>>>>> Sorry for the noise. I also updated the checkstyle step on HBase-TRUNK
>>>>>
>>>>> from
>>>>> mvn checkstyle:checkstyle
>>>>> to
>>>>> mvn -DskipTests package checkstyle:checkstyle
>>>>>
>>>>> to deal with the same issue. Looks all clear now.
>>>>>
>>>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com>
>>>>> wrote:
>>>>>
>>>>>> Updated the following builds:
>>>>>>
>>>>>> * HBase-TRUNK
>>>>>>
>>>>>> moved from
>>>>>>
>>>>>> mvn clean compile findbugs:findbugs
>>>>>>
>>>>>> to
>>>>>>
>>>>>> mvn clean -DskipTests package findbugs:findbugs
>>>>>>
>>>>>> To work around the known issue where we can't do a bootstrap compile
>>>>>> without previous install or remote SNAPSHOT artifacts. (recently triggered
>>>>>> by the addition of the procedure module on master)
>>>>>>
>>>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
>>>>>>
>>>>>>> Make findbugs and checkstyle plugins always run.
>>>>>>> The default behavior only publishes result for stable builds, but
>>>>>>> master is
>>>>>>> not always stable and sometimes we introduce new warnings in unstable
>>>>>>> builds(see builds 6310-6314).
>>>>>>> And findbugs and checkstyle will not fail unless we abort the
>>>>>>> building
>>>>>>> process I think, so it is safe to turn it on every time.
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>>>>>>>
>>>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>>>>>>> >
>>>>>>> > Cheers
>>>>>>> >
>>>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com> wrote:
>>>>>>> >
>>>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
>>>>>>> findbugs,
>>>>>>> > > checkstyle and jacoco coverage report.
>>>>>>> > > The config has been tested on
>>>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30
>>>>>>> times.
>>>>>>> > Not
>>>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
>>>>>>> overhead
>>>>>>> > of
>>>>>>> > > collecting information for code coverage).
>>>>>>> > > Thanks.
>>>>>>> > >
>>>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>>>>>>> > >
>>>>>>> > > > +1, thanks a lot for improving our build hygiene.
>>>>>>> > > >
>>>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <
>>>>>>> enis.soz@gmail.com>
>>>>>>> > > wrote:
>>>>>>> > > >
>>>>>>> > > > > Thanks Sean. This is great.
>>>>>>> > > > >
>>>>>>> > > > > Enis
>>>>>>> > > > >
>>>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <
>>>>>>> busbey@cloudera.com>
>>>>>>> > > > wrote:
>>>>>>> > > > >
>>>>>>> > > > > > FYI I just finished chasing down the breakage for mvn site
>>>>>>> on all
>>>>>>> > > patch
>>>>>>> > > > > > builds.
>>>>>>> > > > > >
>>>>>>> > > > > > HBASE-13191 consolidates the few places in the test-patch
>>>>>>> code
>>>>>>> > where
>>>>>>> > > we
>>>>>>> > > > > > hard coded MAVEN_OPTS.
>>>>>>> > > > > >
>>>>>>> > > > > > If you look at the PreCommit job now, we use the "set
>>>>>>> environment
>>>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything
>>>>>>> else
>>>>>>> > respects
>>>>>>> > > > > that
>>>>>>> > > > > > setting.
>>>>>>> > > > > >
>>>>>>> > > > > > I set the initial value to be a combination of the memory
>>>>>>> > limitations
>>>>>>> > > > > we've
>>>>>>> > > > > > been actually running with (the ~6G was getting ignored)
>>>>>>> and the
>>>>>>> > > > permgen
>>>>>>> > > > > > needed for site.
>>>>>>> > > > > >
>>>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
>>>>>>> > > > > >
>>>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net>
>>>>>>> wrote:
>>>>>>> > > > > >
>>>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
>>>>>>> patch-build
>>>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to
>>>>>>> be the
>>>>>>> > > same
>>>>>>> > > > as
>>>>>>> > > > > > > those of trunk build too, setting
>>>>>>> MAVEN_OPTS="-Xmx6100m"... it
>>>>>>> > had
>>>>>>> > > > been
>>>>>>> > > > > > > 3000.
>>>>>>> > > > > > >
>>>>>>> > > > > > > Yours,
>>>>>>> > > > > > > St.Ack
>>>>>>> > > > > > >
>>>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net>
>>>>>>> wrote:
>>>>>>> > > > > > >
>>>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds and
>>>>>>> or last
>>>>>>> > 7
>>>>>>> > > > > days,
>>>>>>> > > > > > > > whichever comes first.
>>>>>>> > > > > > > > St.Ack
>>>>>>> > > > > > > >
>>>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <
>>>>>>> stack@duboce.net>
>>>>>>> > wrote:
>>>>>>> > > > > > > >
>>>>>>> > > > > > > >> Branch-1 and master have stabilized and now run
>>>>>>> mostly blue
>>>>>>> > > (give
>>>>>>> > > > or
>>>>>>> > > > > > > take
>>>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue
>>>>>>> branch-1 has
>>>>>>> > > helped
>>>>>>> > > > us
>>>>>>> > > > > > > >> identify at least one destabilizing commit in the
>>>>>>> last few
>>>>>>> > days,
>>>>>>> > > > > maybe
>>>>>>> > > > > > > two;
>>>>>>> > > > > > > >> this is as it should be (smile).
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch,
>>>>>>> make sure
>>>>>>> > > > > subsequent
>>>>>>> > > > > > > >> builds stay blue. You can subscribe to
>>>>>>> > builds@hbase.apache.org
>>>>>>> > > to
>>>>>>> > > > > get
>>>>>>> > > > > > > >> notice of failures if not already subscribed.
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >> Thanks,
>>>>>>> > > > > > > >> St.Ack
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >> 1.
>>>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>>>>>>> > > > > > > >> 2.
>>>>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <
>>>>>>> stack@duboce.net>
>>>>>>> > > wrote:
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >>> A few notes on testing.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> Too long to read, infra is more capable now and
>>>>>>> after some
>>>>>>> > > work,
>>>>>>> > > > we
>>>>>>> > > > > > are
>>>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets
>>>>>>> try and
>>>>>>> > > keep
>>>>>>> > > > it
>>>>>>> > > > > > > this
>>>>>>> > > > > > > >>> way going forward.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> A recent spurt of test fixing combined with more
>>>>>>> capable
>>>>>>> > > hardware
>>>>>>> > > > > > seems
>>>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
>>>>>>> passing
>>>>>>> > now
>>>>>>> > > on
>>>>>>> > > > > > > branch-1
>>>>>>> > > > > > > >>> and master. Lets try and keep it this way and start
>>>>>>> to trust
>>>>>>> > > our
>>>>>>> > > > > > test
>>>>>>> > > > > > > runs
>>>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and
>>>>>>> nail them.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> Our tests now run in parallel with other test suites
>>>>>>> where
>>>>>>> > > > previous
>>>>>>> > > > > > we
>>>>>>> > > > > > > >>> ran alone. You can see this sometimes when our zombie
>>>>>>> > detector
>>>>>>> > > > > > reports
>>>>>>> > > > > > > >>> tests from another project altogether as lingerers
>>>>>>> (To be
>>>>>>> > > fixed).
>>>>>>> > > > > > > Some of
>>>>>>> > > > > > > >>> our tests are failing because a concurrent hbase run
>>>>>>> is
>>>>>>> > undoing
>>>>>>> > > > > > > classes and
>>>>>>> > > > > > > >>> data from under it. Also, lets fix.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them to
>>>>>>> > complete.
>>>>>>> > > > > Many
>>>>>>> > > > > > > >>> are heavy-duty integration tests starting up multiple
>>>>>>> > clusters
>>>>>>> > > > and
>>>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they
>>>>>>> pass at
>>>>>>> > all.
>>>>>>> > > > > > > Usually
>>>>>>> > > > > > > >>> integration tests have been cast as unit tests
>>>>>>> because there
>>>>>>> > > was
>>>>>>> > > > no
>>>>>>> > > > > > > where
>>>>>>> > > > > > > >>> else for them to get an airing. We have the
>>>>>>> hbase-it suite
>>>>>>> > now
>>>>>>> > > > > which
>>>>>>> > > > > > > would
>>>>>>> > > > > > > >>> be a more apt place but until these are run on a
>>>>>>> regular
>>>>>>> > basis
>>>>>>> > > in
>>>>>>> > > > > > > public
>>>>>>> > > > > > > >>> for all to see, the fat integration tests disguised
>>>>>>> as unit
>>>>>>> > > tests
>>>>>>> > > > > > will
>>>>>>> > > > > > > >>> remain. A review of our current unit tests weeding
>>>>>>> the old
>>>>>>> > > cruft
>>>>>>> > > > > and
>>>>>>> > > > > > > the
>>>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
>>>>>>> undertaking
>>>>>>> > if
>>>>>>> > > > > > > someone is
>>>>>>> > > > > > > >>> looking to contribute.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> Alex Newman has been working on making our tests
>>>>>>> work up on
>>>>>>> > > > travis
>>>>>>> > > > > > and
>>>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes
>>>>>>> end-to-end. He
>>>>>>> > also
>>>>>>> > > > > added
>>>>>>> > > > > > in
>>>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
>>>>>>> mapreduce --
>>>>>>> > > > > alongside
>>>>>>> > > > > > > our
>>>>>>> > > > > > > >>> old "sizing" categorizations of small/medium/large.
>>>>>>> His
>>>>>>> > > thinking
>>>>>>> > > > > is
>>>>>>> > > > > > > that
>>>>>>> > > > > > > >>> we can run these categorizations in parallel so we
>>>>>>> could run
>>>>>>> > > the
>>>>>>> > > > > > total
>>>>>>> > > > > > > >>> suite in about the time of the longest test, say
>>>>>>> > 20-30minutes?
>>>>>>> > > > We
>>>>>>> > > > > > > could
>>>>>>> > > > > > > >>> even change Apache to run them this way.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> FYI,
>>>>>>> > > > > > > >>> St.Ack
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >
>>>>>>> > > > > > >
>>>>>>> > > > > >
>>>>>>> > > > > >
>>>>>>> > > > > >
>>>>>>> > > > > > --
>>>>>>> > > > > > Sean
>>>>>>> > > > > >
>>>>>>> > > > >
>>>>>>> > > >
>>>>>>> > > >
>>>>>>> > > >
>>>>>>> > > > --
>>>>>>> > > > Best regards,
>>>>>>> > > >
>>>>>>> > > > - Andy
>>>>>>> > > >
>>>>>>> > > > Problems worthy of attack prove their worth by hitting back. -
>>>>>>> Piet
>>>>>>> > Hein
>>>>>>> > > > (via Tom White)
>>>>>>> > > >
>>>>>>> > >
>>>>>>> >
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sean
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sean
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sean
>>>>
>>>
>>>
>>>
>>> --
>>> Sean
>>>
>>
>>
>>
>> --
>> Sean
>>
>
>
>
> --
> Sean
>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
I moved TestBulkIngest and TestIngest out to HBASE-13952 and HBASE-13953,
respectively.
added versions of the remaining tests that runs on branch-1 and master.
presuming things come back passing I'll turn on notifications tomorrow.
On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com> wrote:
> I've add a new build test to run the ITs under sunny day conditions using
> minicluster for both java 7 and java 8 on the 1.2 line.
>
> https://builds.apache.org/job/HBase-1.2-IT/
>
> I haven't turned on notifications yet, because BulkIngest and TestIngest
> are read. details on HBASE-13750
>
> On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> sigh. that should have been "is now a matrix build".
>>
>> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com> wrote:
>>
>>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 in
>>> parallel.
>>>
>>> I saved the old job for now and left it disabled[2].
>>>
>>>
>>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
>>> [2]:
>>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>>>
>>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com>
>>> wrote:
>>>
>>>> Sorry for the noise. I also updated the checkstyle step on HBase-TRUNK
>>>>
>>>> from
>>>> mvn checkstyle:checkstyle
>>>> to
>>>> mvn -DskipTests package checkstyle:checkstyle
>>>>
>>>> to deal with the same issue. Looks all clear now.
>>>>
>>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com>
>>>> wrote:
>>>>
>>>>> Updated the following builds:
>>>>>
>>>>> * HBase-TRUNK
>>>>>
>>>>> moved from
>>>>>
>>>>> mvn clean compile findbugs:findbugs
>>>>>
>>>>> to
>>>>>
>>>>> mvn clean -DskipTests package findbugs:findbugs
>>>>>
>>>>> To work around the known issue where we can't do a bootstrap compile
>>>>> without previous install or remote SNAPSHOT artifacts. (recently triggered
>>>>> by the addition of the procedure module on master)
>>>>>
>>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
>>>>>
>>>>>> Make findbugs and checkstyle plugins always run.
>>>>>> The default behavior only publishes result for stable builds, but
>>>>>> master is
>>>>>> not always stable and sometimes we introduce new warnings in unstable
>>>>>> builds(see builds 6310-6314).
>>>>>> And findbugs and checkstyle will not fail unless we abort the building
>>>>>> process I think, so it is safe to turn it on every time.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>>>>>>
>>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>>>>>> >
>>>>>> > Cheers
>>>>>> >
>>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com> wrote:
>>>>>> >
>>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
>>>>>> findbugs,
>>>>>> > > checkstyle and jacoco coverage report.
>>>>>> > > The config has been tested on
>>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30
>>>>>> times.
>>>>>> > Not
>>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
>>>>>> overhead
>>>>>> > of
>>>>>> > > collecting information for code coverage).
>>>>>> > > Thanks.
>>>>>> > >
>>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>>>>>> > >
>>>>>> > > > +1, thanks a lot for improving our build hygiene.
>>>>>> > > >
>>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <
>>>>>> enis.soz@gmail.com>
>>>>>> > > wrote:
>>>>>> > > >
>>>>>> > > > > Thanks Sean. This is great.
>>>>>> > > > >
>>>>>> > > > > Enis
>>>>>> > > > >
>>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <
>>>>>> busbey@cloudera.com>
>>>>>> > > > wrote:
>>>>>> > > > >
>>>>>> > > > > > FYI I just finished chasing down the breakage for mvn site
>>>>>> on all
>>>>>> > > patch
>>>>>> > > > > > builds.
>>>>>> > > > > >
>>>>>> > > > > > HBASE-13191 consolidates the few places in the test-patch
>>>>>> code
>>>>>> > where
>>>>>> > > we
>>>>>> > > > > > hard coded MAVEN_OPTS.
>>>>>> > > > > >
>>>>>> > > > > > If you look at the PreCommit job now, we use the "set
>>>>>> environment
>>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything else
>>>>>> > respects
>>>>>> > > > > that
>>>>>> > > > > > setting.
>>>>>> > > > > >
>>>>>> > > > > > I set the initial value to be a combination of the memory
>>>>>> > limitations
>>>>>> > > > > we've
>>>>>> > > > > > been actually running with (the ~6G was getting ignored)
>>>>>> and the
>>>>>> > > > permgen
>>>>>> > > > > > needed for site.
>>>>>> > > > > >
>>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
>>>>>> > > > > >
>>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net>
>>>>>> wrote:
>>>>>> > > > > >
>>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
>>>>>> patch-build
>>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to
>>>>>> be the
>>>>>> > > same
>>>>>> > > > as
>>>>>> > > > > > > those of trunk build too, setting
>>>>>> MAVEN_OPTS="-Xmx6100m"... it
>>>>>> > had
>>>>>> > > > been
>>>>>> > > > > > > 3000.
>>>>>> > > > > > >
>>>>>> > > > > > > Yours,
>>>>>> > > > > > > St.Ack
>>>>>> > > > > > >
>>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net>
>>>>>> wrote:
>>>>>> > > > > > >
>>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds and
>>>>>> or last
>>>>>> > 7
>>>>>> > > > > days,
>>>>>> > > > > > > > whichever comes first.
>>>>>> > > > > > > > St.Ack
>>>>>> > > > > > > >
>>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <stack@duboce.net
>>>>>> >
>>>>>> > wrote:
>>>>>> > > > > > > >
>>>>>> > > > > > > >> Branch-1 and master have stabilized and now run mostly
>>>>>> blue
>>>>>> > > (give
>>>>>> > > > or
>>>>>> > > > > > > take
>>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1
>>>>>> has
>>>>>> > > helped
>>>>>> > > > us
>>>>>> > > > > > > >> identify at least one destabilizing commit in the last
>>>>>> few
>>>>>> > days,
>>>>>> > > > > maybe
>>>>>> > > > > > > two;
>>>>>> > > > > > > >> this is as it should be (smile).
>>>>>> > > > > > > >>
>>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch, make
>>>>>> sure
>>>>>> > > > > subsequent
>>>>>> > > > > > > >> builds stay blue. You can subscribe to
>>>>>> > builds@hbase.apache.org
>>>>>> > > to
>>>>>> > > > > get
>>>>>> > > > > > > >> notice of failures if not already subscribed.
>>>>>> > > > > > > >>
>>>>>> > > > > > > >> Thanks,
>>>>>> > > > > > > >> St.Ack
>>>>>> > > > > > > >>
>>>>>> > > > > > > >> 1.
>>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>>>>>> > > > > > > >> 2.
>>>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>>>>>> > > > > > > >>
>>>>>> > > > > > > >>
>>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <
>>>>>> stack@duboce.net>
>>>>>> > > wrote:
>>>>>> > > > > > > >>
>>>>>> > > > > > > >>> A few notes on testing.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> Too long to read, infra is more capable now and after
>>>>>> some
>>>>>> > > work,
>>>>>> > > > we
>>>>>> > > > > > are
>>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets
>>>>>> try and
>>>>>> > > keep
>>>>>> > > > it
>>>>>> > > > > > > this
>>>>>> > > > > > > >>> way going forward.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> A recent spurt of test fixing combined with more
>>>>>> capable
>>>>>> > > hardware
>>>>>> > > > > > seems
>>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
>>>>>> passing
>>>>>> > now
>>>>>> > > on
>>>>>> > > > > > > branch-1
>>>>>> > > > > > > >>> and master. Lets try and keep it this way and start
>>>>>> to trust
>>>>>> > > our
>>>>>> > > > > > test
>>>>>> > > > > > > runs
>>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and nail
>>>>>> them.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> Our tests now run in parallel with other test suites
>>>>>> where
>>>>>> > > > previous
>>>>>> > > > > > we
>>>>>> > > > > > > >>> ran alone. You can see this sometimes when our zombie
>>>>>> > detector
>>>>>> > > > > > reports
>>>>>> > > > > > > >>> tests from another project altogether as lingerers
>>>>>> (To be
>>>>>> > > fixed).
>>>>>> > > > > > > Some of
>>>>>> > > > > > > >>> our tests are failing because a concurrent hbase run
>>>>>> is
>>>>>> > undoing
>>>>>> > > > > > > classes and
>>>>>> > > > > > > >>> data from under it. Also, lets fix.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them to
>>>>>> > complete.
>>>>>> > > > > Many
>>>>>> > > > > > > >>> are heavy-duty integration tests starting up multiple
>>>>>> > clusters
>>>>>> > > > and
>>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they
>>>>>> pass at
>>>>>> > all.
>>>>>> > > > > > > Usually
>>>>>> > > > > > > >>> integration tests have been cast as unit tests
>>>>>> because there
>>>>>> > > was
>>>>>> > > > no
>>>>>> > > > > > > where
>>>>>> > > > > > > >>> else for them to get an airing. We have the hbase-it
>>>>>> suite
>>>>>> > now
>>>>>> > > > > which
>>>>>> > > > > > > would
>>>>>> > > > > > > >>> be a more apt place but until these are run on a
>>>>>> regular
>>>>>> > basis
>>>>>> > > in
>>>>>> > > > > > > public
>>>>>> > > > > > > >>> for all to see, the fat integration tests disguised
>>>>>> as unit
>>>>>> > > tests
>>>>>> > > > > > will
>>>>>> > > > > > > >>> remain. A review of our current unit tests weeding
>>>>>> the old
>>>>>> > > cruft
>>>>>> > > > > and
>>>>>> > > > > > > the
>>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
>>>>>> undertaking
>>>>>> > if
>>>>>> > > > > > > someone is
>>>>>> > > > > > > >>> looking to contribute.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> Alex Newman has been working on making our tests work
>>>>>> up on
>>>>>> > > > travis
>>>>>> > > > > > and
>>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes
>>>>>> end-to-end. He
>>>>>> > also
>>>>>> > > > > added
>>>>>> > > > > > in
>>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
>>>>>> mapreduce --
>>>>>> > > > > alongside
>>>>>> > > > > > > our
>>>>>> > > > > > > >>> old "sizing" categorizations of small/medium/large.
>>>>>> His
>>>>>> > > thinking
>>>>>> > > > > is
>>>>>> > > > > > > that
>>>>>> > > > > > > >>> we can run these categorizations in parallel so we
>>>>>> could run
>>>>>> > > the
>>>>>> > > > > > total
>>>>>> > > > > > > >>> suite in about the time of the longest test, say
>>>>>> > 20-30minutes?
>>>>>> > > > We
>>>>>> > > > > > > could
>>>>>> > > > > > > >>> even change Apache to run them this way.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> FYI,
>>>>>> > > > > > > >>> St.Ack
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>
>>>>>> > > > > > > >
>>>>>> > > > > > >
>>>>>> > > > > >
>>>>>> > > > > >
>>>>>> > > > > >
>>>>>> > > > > > --
>>>>>> > > > > > Sean
>>>>>> > > > > >
>>>>>> > > > >
>>>>>> > > >
>>>>>> > > >
>>>>>> > > >
>>>>>> > > > --
>>>>>> > > > Best regards,
>>>>>> > > >
>>>>>> > > > - Andy
>>>>>> > > >
>>>>>> > > > Problems worthy of attack prove their worth by hitting back. -
>>>>>> Piet
>>>>>> > Hein
>>>>>> > > > (via Tom White)
>>>>>> > > >
>>>>>> > >
>>>>>> >
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sean
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sean
>>>>
>>>
>>>
>>>
>>> --
>>> Sean
>>>
>>
>>
>>
>> --
>> Sean
>>
>
>
>
> --
> Sean
>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
Please see this note on hadoop-common about an infra issue that's causing a
temporary delay:
*http://s.apache.org/6XU <http://s.apache.org/6XU>*
On Fri, Jun 19, 2015 at 7:31 PM, Enis Söztutar <en...@gmail.com> wrote:
> It seems that precommit-admin job which triggers the precommit-hbase job is
> not running as frequently as it used to be. Only once every ~6 hours or so.
>
> https://builds.apache.org/job/PreCommit-Admin/
>
> The job is starved for the slaves.
>
> Enis
>
> On Fri, Jun 19, 2015 at 5:23 PM, Enis Söztutar <en...@gmail.com> wrote:
>
> > hadoopqa is acting weird for the last couple of days. It does not report
> > back the results. One example is:
> > https://builds.apache.org/job/PreCommit-HBASE-Build/14475/console.
> >
> > I've excluded ubuntu3 from the list because it does not have protoc-2.5.
> >
> >
> > On Mon, Jun 15, 2015 at 9:55 PM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> >> Added new HBase-1.3 build or branch-1 and changed the existing 1.2
> builds
> >> to use branch-1.2. (ref HBASE-13912)
> >>
> >> On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >>
> >> > I've add a new build test to run the ITs under sunny day conditions
> >> using
> >> > minicluster for both java 7 and java 8 on the 1.2 line.
> >> >
> >> > https://builds.apache.org/job/HBase-1.2-IT/
> >> >
> >> > I haven't turned on notifications yet, because BulkIngest and
> TestIngest
> >> > are read. details on HBASE-13750
> >> >
> >> > On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com>
> >> wrote:
> >> >
> >> >> sigh. that should have been "is now a matrix build".
> >> >>
> >> >> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com>
> >> wrote:
> >> >>
> >> >>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8
> >> in
> >> >>> parallel.
> >> >>>
> >> >>> I saved the old job for now and left it disabled[2].
> >> >>>
> >> >>>
> >> >>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
> >> >>> [2]:
> >> >>>
> >>
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
> >> >>>
> >> >>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com>
> >> >>> wrote:
> >> >>>
> >> >>>> Sorry for the noise. I also updated the checkstyle step on
> >> HBase-TRUNK
> >> >>>>
> >> >>>> from
> >> >>>> mvn checkstyle:checkstyle
> >> >>>> to
> >> >>>> mvn -DskipTests package checkstyle:checkstyle
> >> >>>>
> >> >>>> to deal with the same issue. Looks all clear now.
> >> >>>>
> >> >>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com>
> >> >>>> wrote:
> >> >>>>
> >> >>>>> Updated the following builds:
> >> >>>>>
> >> >>>>> * HBase-TRUNK
> >> >>>>>
> >> >>>>> moved from
> >> >>>>>
> >> >>>>> mvn clean compile findbugs:findbugs
> >> >>>>>
> >> >>>>> to
> >> >>>>>
> >> >>>>> mvn clean -DskipTests package findbugs:findbugs
> >> >>>>>
> >> >>>>> To work around the known issue where we can't do a bootstrap
> compile
> >> >>>>> without previous install or remote SNAPSHOT artifacts. (recently
> >> triggered
> >> >>>>> by the addition of the procedure module on master)
> >> >>>>>
> >> >>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com>
> wrote:
> >> >>>>>
> >> >>>>>> Make findbugs and checkstyle plugins always run.
> >> >>>>>> The default behavior only publishes result for stable builds, but
> >> >>>>>> master is
> >> >>>>>> not always stable and sometimes we introduce new warnings in
> >> unstable
> >> >>>>>> builds(see builds 6310-6314).
> >> >>>>>> And findbugs and checkstyle will not fail unless we abort the
> >> building
> >> >>>>>> process I think, so it is safe to turn it on every time.
> >> >>>>>>
> >> >>>>>> Thanks.
> >> >>>>>>
> >> >>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
> >> >>>>>>
> >> >>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
> >> >>>>>> >
> >> >>>>>> > Cheers
> >> >>>>>> >
> >> >>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com>
> >> wrote:
> >> >>>>>> >
> >> >>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
> >> >>>>>> findbugs,
> >> >>>>>> > > checkstyle and jacoco coverage report.
> >> >>>>>> > > The config has been tested on
> >> >>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly
> 30
> >> >>>>>> times.
> >> >>>>>> > Not
> >> >>>>>> > > much different from HBase-TRUNK unless it runs ~30%
> slower(the
> >> >>>>>> overhead
> >> >>>>>> > of
> >> >>>>>> > > collecting information for code coverage).
> >> >>>>>> > > Thanks.
> >> >>>>>> > >
> >> >>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <
> apurtell@apache.org
> >> >:
> >> >>>>>> > >
> >> >>>>>> > > > +1, thanks a lot for improving our build hygiene.
> >> >>>>>> > > >
> >> >>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <
> >> >>>>>> enis.soz@gmail.com>
> >> >>>>>> > > wrote:
> >> >>>>>> > > >
> >> >>>>>> > > > > Thanks Sean. This is great.
> >> >>>>>> > > > >
> >> >>>>>> > > > > Enis
> >> >>>>>> > > > >
> >> >>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <
> >> >>>>>> busbey@cloudera.com>
> >> >>>>>> > > > wrote:
> >> >>>>>> > > > >
> >> >>>>>> > > > > > FYI I just finished chasing down the breakage for mvn
> >> site
> >> >>>>>> on all
> >> >>>>>> > > patch
> >> >>>>>> > > > > > builds.
> >> >>>>>> > > > > >
> >> >>>>>> > > > > > HBASE-13191 consolidates the few places in the
> test-patch
> >> >>>>>> code
> >> >>>>>> > where
> >> >>>>>> > > we
> >> >>>>>> > > > > > hard coded MAVEN_OPTS.
> >> >>>>>> > > > > >
> >> >>>>>> > > > > > If you look at the PreCommit job now, we use the "set
> >> >>>>>> environment
> >> >>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything
> >> else
> >> >>>>>> > respects
> >> >>>>>> > > > > that
> >> >>>>>> > > > > > setting.
> >> >>>>>> > > > > >
> >> >>>>>> > > > > > I set the initial value to be a combination of the
> memory
> >> >>>>>> > limitations
> >> >>>>>> > > > > we've
> >> >>>>>> > > > > > been actually running with (the ~6G was getting
> ignored)
> >> >>>>>> and the
> >> >>>>>> > > > permgen
> >> >>>>>> > > > > > needed for site.
> >> >>>>>> > > > > >
> >> >>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData
> >> -XX:MaxPermSize=256m
> >> >>>>>> > > > > >
> >> >>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <
> >> stack@duboce.net>
> >> >>>>>> wrote:
> >> >>>>>> > > > > >
> >> >>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
> >> >>>>>> patch-build
> >> >>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS
> >> to
> >> >>>>>> be the
> >> >>>>>> > > same
> >> >>>>>> > > > as
> >> >>>>>> > > > > > > those of trunk build too, setting
> >> >>>>>> MAVEN_OPTS="-Xmx6100m"... it
> >> >>>>>> > had
> >> >>>>>> > > > been
> >> >>>>>> > > > > > > 3000.
> >> >>>>>> > > > > > >
> >> >>>>>> > > > > > > Yours,
> >> >>>>>> > > > > > > St.Ack
> >> >>>>>> > > > > > >
> >> >>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <
> >> stack@duboce.net>
> >> >>>>>> wrote:
> >> >>>>>> > > > > > >
> >> >>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds
> >> and
> >> >>>>>> or last
> >> >>>>>> > 7
> >> >>>>>> > > > > days,
> >> >>>>>> > > > > > > > whichever comes first.
> >> >>>>>> > > > > > > > St.Ack
> >> >>>>>> > > > > > > >
> >> >>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <
> >> stack@duboce.net
> >> >>>>>> >
> >> >>>>>> > wrote:
> >> >>>>>> > > > > > > >
> >> >>>>>> > > > > > > >> Branch-1 and master have stabilized and now run
> >> mostly
> >> >>>>>> blue
> >> >>>>>> > > (give
> >> >>>>>> > > > or
> >> >>>>>> > > > > > > take
> >> >>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue
> >> branch-1
> >> >>>>>> has
> >> >>>>>> > > helped
> >> >>>>>> > > > us
> >> >>>>>> > > > > > > >> identify at least one destabilizing commit in the
> >> last
> >> >>>>>> few
> >> >>>>>> > days,
> >> >>>>>> > > > > maybe
> >> >>>>>> > > > > > > two;
> >> >>>>>> > > > > > > >> this is as it should be (smile).
> >> >>>>>> > > > > > > >>
> >> >>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch,
> >> make
> >> >>>>>> sure
> >> >>>>>> > > > > subsequent
> >> >>>>>> > > > > > > >> builds stay blue. You can subscribe to
> >> >>>>>> > builds@hbase.apache.org
> >> >>>>>> > > to
> >> >>>>>> > > > > get
> >> >>>>>> > > > > > > >> notice of failures if not already subscribed.
> >> >>>>>> > > > > > > >>
> >> >>>>>> > > > > > > >> Thanks,
> >> >>>>>> > > > > > > >> St.Ack
> >> >>>>>> > > > > > > >>
> >> >>>>>> > > > > > > >> 1.
> >> >>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> >> >>>>>> > > > > > > >> 2.
> >> >>>>>> > >
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> >> >>>>>> > > > > > > >>
> >> >>>>>> > > > > > > >>
> >> >>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <
> >> >>>>>> stack@duboce.net>
> >> >>>>>> > > wrote:
> >> >>>>>> > > > > > > >>
> >> >>>>>> > > > > > > >>> A few notes on testing.
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>> Too long to read, infra is more capable now and
> >> after
> >> >>>>>> some
> >> >>>>>> > > work,
> >> >>>>>> > > > we
> >> >>>>>> > > > > > are
> >> >>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue.
> Lets
> >> >>>>>> try and
> >> >>>>>> > > keep
> >> >>>>>> > > > it
> >> >>>>>> > > > > > > this
> >> >>>>>> > > > > > > >>> way going forward.
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>> A recent spurt of test fixing combined with more
> >> >>>>>> capable
> >> >>>>>> > > hardware
> >> >>>>>> > > > > > seems
> >> >>>>>> > > > > > > >>> to have gotten us to a new place; tests are
> mostly
> >> >>>>>> passing
> >> >>>>>> > now
> >> >>>>>> > > on
> >> >>>>>> > > > > > > branch-1
> >> >>>>>> > > > > > > >>> and master. Lets try and keep it this way and
> >> start
> >> >>>>>> to trust
> >> >>>>>> > > our
> >> >>>>>> > > > > > test
> >> >>>>>> > > > > > > runs
> >> >>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and
> >> nail
> >> >>>>>> them.
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>> Our tests now run in parallel with other test
> >> suites
> >> >>>>>> where
> >> >>>>>> > > > previous
> >> >>>>>> > > > > > we
> >> >>>>>> > > > > > > >>> ran alone. You can see this sometimes when our
> >> zombie
> >> >>>>>> > detector
> >> >>>>>> > > > > > reports
> >> >>>>>> > > > > > > >>> tests from another project altogether as
> lingerers
> >> >>>>>> (To be
> >> >>>>>> > > fixed).
> >> >>>>>> > > > > > > Some of
> >> >>>>>> > > > > > > >>> our tests are failing because a concurrent hbase
> >> run
> >> >>>>>> is
> >> >>>>>> > undoing
> >> >>>>>> > > > > > > classes and
> >> >>>>>> > > > > > > >>> data from under it. Also, lets fix.
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for
> them
> >> to
> >> >>>>>> > complete.
> >> >>>>>> > > > > Many
> >> >>>>>> > > > > > > >>> are heavy-duty integration tests starting up
> >> multiple
> >> >>>>>> > clusters
> >> >>>>>> > > > and
> >> >>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle
> they
> >> >>>>>> pass at
> >> >>>>>> > all.
> >> >>>>>> > > > > > > Usually
> >> >>>>>> > > > > > > >>> integration tests have been cast as unit tests
> >> >>>>>> because there
> >> >>>>>> > > was
> >> >>>>>> > > > no
> >> >>>>>> > > > > > > where
> >> >>>>>> > > > > > > >>> else for them to get an airing. We have the
> >> hbase-it
> >> >>>>>> suite
> >> >>>>>> > now
> >> >>>>>> > > > > which
> >> >>>>>> > > > > > > would
> >> >>>>>> > > > > > > >>> be a more apt place but until these are run on a
> >> >>>>>> regular
> >> >>>>>> > basis
> >> >>>>>> > > in
> >> >>>>>> > > > > > > public
> >> >>>>>> > > > > > > >>> for all to see, the fat integration tests
> disguised
> >> >>>>>> as unit
> >> >>>>>> > > tests
> >> >>>>>> > > > > > will
> >> >>>>>> > > > > > > >>> remain. A review of our current unit tests
> weeding
> >> >>>>>> the old
> >> >>>>>> > > cruft
> >> >>>>>> > > > > and
> >> >>>>>> > > > > > > the
> >> >>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
> >> >>>>>> undertaking
> >> >>>>>> > if
> >> >>>>>> > > > > > > someone is
> >> >>>>>> > > > > > > >>> looking to contribute.
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>> Alex Newman has been working on making our tests
> >> work
> >> >>>>>> up on
> >> >>>>>> > > > travis
> >> >>>>>> > > > > > and
> >> >>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes
> >> >>>>>> end-to-end. He
> >> >>>>>> > also
> >> >>>>>> > > > > added
> >> >>>>>> > > > > > in
> >> >>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
> >> >>>>>> mapreduce --
> >> >>>>>> > > > > alongside
> >> >>>>>> > > > > > > our
> >> >>>>>> > > > > > > >>> old "sizing" categorizations of
> small/medium/large.
> >> >>>>>> His
> >> >>>>>> > > thinking
> >> >>>>>> > > > > is
> >> >>>>>> > > > > > > that
> >> >>>>>> > > > > > > >>> we can run these categorizations in parallel so
> we
> >> >>>>>> could run
> >> >>>>>> > > the
> >> >>>>>> > > > > > total
> >> >>>>>> > > > > > > >>> suite in about the time of the longest test, say
> >> >>>>>> > 20-30minutes?
> >> >>>>>> > > > We
> >> >>>>>> > > > > > > could
> >> >>>>>> > > > > > > >>> even change Apache to run them this way.
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>> FYI,
> >> >>>>>> > > > > > > >>> St.Ack
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>>
> >> >>>>>> > > > > > > >>
> >> >>>>>> > > > > > > >
> >> >>>>>> > > > > > >
> >> >>>>>> > > > > >
> >> >>>>>> > > > > >
> >> >>>>>> > > > > >
> >> >>>>>> > > > > > --
> >> >>>>>> > > > > > Sean
> >> >>>>>> > > > > >
> >> >>>>>> > > > >
> >> >>>>>> > > >
> >> >>>>>> > > >
> >> >>>>>> > > >
> >> >>>>>> > > > --
> >> >>>>>> > > > Best regards,
> >> >>>>>> > > >
> >> >>>>>> > > > - Andy
> >> >>>>>> > > >
> >> >>>>>> > > > Problems worthy of attack prove their worth by hitting
> back.
> >> -
> >> >>>>>> Piet
> >> >>>>>> > Hein
> >> >>>>>> > > > (via Tom White)
> >> >>>>>> > > >
> >> >>>>>> > >
> >> >>>>>> >
> >> >>>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>> --
> >> >>>>> Sean
> >> >>>>>
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> --
> >> >>>> Sean
> >> >>>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Sean
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Sean
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Sean
> >> >
> >>
> >>
> >>
> >> --
> >> Sean
> >>
> >
> >
>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Enis Söztutar <en...@gmail.com>.
It seems that precommit-admin job which triggers the precommit-hbase job is
not running as frequently as it used to be. Only once every ~6 hours or so.
https://builds.apache.org/job/PreCommit-Admin/
The job is starved for the slaves.
Enis
On Fri, Jun 19, 2015 at 5:23 PM, Enis Söztutar <en...@gmail.com> wrote:
> hadoopqa is acting weird for the last couple of days. It does not report
> back the results. One example is:
> https://builds.apache.org/job/PreCommit-HBASE-Build/14475/console.
>
> I've excluded ubuntu3 from the list because it does not have protoc-2.5.
>
>
> On Mon, Jun 15, 2015 at 9:55 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> Added new HBase-1.3 build or branch-1 and changed the existing 1.2 builds
>> to use branch-1.2. (ref HBASE-13912)
>>
>> On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com> wrote:
>>
>> > I've add a new build test to run the ITs under sunny day conditions
>> using
>> > minicluster for both java 7 and java 8 on the 1.2 line.
>> >
>> > https://builds.apache.org/job/HBase-1.2-IT/
>> >
>> > I haven't turned on notifications yet, because BulkIngest and TestIngest
>> > are read. details on HBASE-13750
>> >
>> > On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com>
>> wrote:
>> >
>> >> sigh. that should have been "is now a matrix build".
>> >>
>> >> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com>
>> wrote:
>> >>
>> >>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8
>> in
>> >>> parallel.
>> >>>
>> >>> I saved the old job for now and left it disabled[2].
>> >>>
>> >>>
>> >>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
>> >>> [2]:
>> >>>
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>> >>>
>> >>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com>
>> >>> wrote:
>> >>>
>> >>>> Sorry for the noise. I also updated the checkstyle step on
>> HBase-TRUNK
>> >>>>
>> >>>> from
>> >>>> mvn checkstyle:checkstyle
>> >>>> to
>> >>>> mvn -DskipTests package checkstyle:checkstyle
>> >>>>
>> >>>> to deal with the same issue. Looks all clear now.
>> >>>>
>> >>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com>
>> >>>> wrote:
>> >>>>
>> >>>>> Updated the following builds:
>> >>>>>
>> >>>>> * HBase-TRUNK
>> >>>>>
>> >>>>> moved from
>> >>>>>
>> >>>>> mvn clean compile findbugs:findbugs
>> >>>>>
>> >>>>> to
>> >>>>>
>> >>>>> mvn clean -DskipTests package findbugs:findbugs
>> >>>>>
>> >>>>> To work around the known issue where we can't do a bootstrap compile
>> >>>>> without previous install or remote SNAPSHOT artifacts. (recently
>> triggered
>> >>>>> by the addition of the procedure module on master)
>> >>>>>
>> >>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
>> >>>>>
>> >>>>>> Make findbugs and checkstyle plugins always run.
>> >>>>>> The default behavior only publishes result for stable builds, but
>> >>>>>> master is
>> >>>>>> not always stable and sometimes we introduce new warnings in
>> unstable
>> >>>>>> builds(see builds 6310-6314).
>> >>>>>> And findbugs and checkstyle will not fail unless we abort the
>> building
>> >>>>>> process I think, so it is safe to turn it on every time.
>> >>>>>>
>> >>>>>> Thanks.
>> >>>>>>
>> >>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>> >>>>>>
>> >>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>> >>>>>> >
>> >>>>>> > Cheers
>> >>>>>> >
>> >>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com>
>> wrote:
>> >>>>>> >
>> >>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
>> >>>>>> findbugs,
>> >>>>>> > > checkstyle and jacoco coverage report.
>> >>>>>> > > The config has been tested on
>> >>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30
>> >>>>>> times.
>> >>>>>> > Not
>> >>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
>> >>>>>> overhead
>> >>>>>> > of
>> >>>>>> > > collecting information for code coverage).
>> >>>>>> > > Thanks.
>> >>>>>> > >
>> >>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <apurtell@apache.org
>> >:
>> >>>>>> > >
>> >>>>>> > > > +1, thanks a lot for improving our build hygiene.
>> >>>>>> > > >
>> >>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <
>> >>>>>> enis.soz@gmail.com>
>> >>>>>> > > wrote:
>> >>>>>> > > >
>> >>>>>> > > > > Thanks Sean. This is great.
>> >>>>>> > > > >
>> >>>>>> > > > > Enis
>> >>>>>> > > > >
>> >>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <
>> >>>>>> busbey@cloudera.com>
>> >>>>>> > > > wrote:
>> >>>>>> > > > >
>> >>>>>> > > > > > FYI I just finished chasing down the breakage for mvn
>> site
>> >>>>>> on all
>> >>>>>> > > patch
>> >>>>>> > > > > > builds.
>> >>>>>> > > > > >
>> >>>>>> > > > > > HBASE-13191 consolidates the few places in the test-patch
>> >>>>>> code
>> >>>>>> > where
>> >>>>>> > > we
>> >>>>>> > > > > > hard coded MAVEN_OPTS.
>> >>>>>> > > > > >
>> >>>>>> > > > > > If you look at the PreCommit job now, we use the "set
>> >>>>>> environment
>> >>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything
>> else
>> >>>>>> > respects
>> >>>>>> > > > > that
>> >>>>>> > > > > > setting.
>> >>>>>> > > > > >
>> >>>>>> > > > > > I set the initial value to be a combination of the memory
>> >>>>>> > limitations
>> >>>>>> > > > > we've
>> >>>>>> > > > > > been actually running with (the ~6G was getting ignored)
>> >>>>>> and the
>> >>>>>> > > > permgen
>> >>>>>> > > > > > needed for site.
>> >>>>>> > > > > >
>> >>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData
>> -XX:MaxPermSize=256m
>> >>>>>> > > > > >
>> >>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <
>> stack@duboce.net>
>> >>>>>> wrote:
>> >>>>>> > > > > >
>> >>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
>> >>>>>> patch-build
>> >>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS
>> to
>> >>>>>> be the
>> >>>>>> > > same
>> >>>>>> > > > as
>> >>>>>> > > > > > > those of trunk build too, setting
>> >>>>>> MAVEN_OPTS="-Xmx6100m"... it
>> >>>>>> > had
>> >>>>>> > > > been
>> >>>>>> > > > > > > 3000.
>> >>>>>> > > > > > >
>> >>>>>> > > > > > > Yours,
>> >>>>>> > > > > > > St.Ack
>> >>>>>> > > > > > >
>> >>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <
>> stack@duboce.net>
>> >>>>>> wrote:
>> >>>>>> > > > > > >
>> >>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds
>> and
>> >>>>>> or last
>> >>>>>> > 7
>> >>>>>> > > > > days,
>> >>>>>> > > > > > > > whichever comes first.
>> >>>>>> > > > > > > > St.Ack
>> >>>>>> > > > > > > >
>> >>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <
>> stack@duboce.net
>> >>>>>> >
>> >>>>>> > wrote:
>> >>>>>> > > > > > > >
>> >>>>>> > > > > > > >> Branch-1 and master have stabilized and now run
>> mostly
>> >>>>>> blue
>> >>>>>> > > (give
>> >>>>>> > > > or
>> >>>>>> > > > > > > take
>> >>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue
>> branch-1
>> >>>>>> has
>> >>>>>> > > helped
>> >>>>>> > > > us
>> >>>>>> > > > > > > >> identify at least one destabilizing commit in the
>> last
>> >>>>>> few
>> >>>>>> > days,
>> >>>>>> > > > > maybe
>> >>>>>> > > > > > > two;
>> >>>>>> > > > > > > >> this is as it should be (smile).
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch,
>> make
>> >>>>>> sure
>> >>>>>> > > > > subsequent
>> >>>>>> > > > > > > >> builds stay blue. You can subscribe to
>> >>>>>> > builds@hbase.apache.org
>> >>>>>> > > to
>> >>>>>> > > > > get
>> >>>>>> > > > > > > >> notice of failures if not already subscribed.
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >> Thanks,
>> >>>>>> > > > > > > >> St.Ack
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >> 1.
>> >>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>> >>>>>> > > > > > > >> 2.
>> >>>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <
>> >>>>>> stack@duboce.net>
>> >>>>>> > > wrote:
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >>> A few notes on testing.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> Too long to read, infra is more capable now and
>> after
>> >>>>>> some
>> >>>>>> > > work,
>> >>>>>> > > > we
>> >>>>>> > > > > > are
>> >>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets
>> >>>>>> try and
>> >>>>>> > > keep
>> >>>>>> > > > it
>> >>>>>> > > > > > > this
>> >>>>>> > > > > > > >>> way going forward.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> A recent spurt of test fixing combined with more
>> >>>>>> capable
>> >>>>>> > > hardware
>> >>>>>> > > > > > seems
>> >>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
>> >>>>>> passing
>> >>>>>> > now
>> >>>>>> > > on
>> >>>>>> > > > > > > branch-1
>> >>>>>> > > > > > > >>> and master. Lets try and keep it this way and
>> start
>> >>>>>> to trust
>> >>>>>> > > our
>> >>>>>> > > > > > test
>> >>>>>> > > > > > > runs
>> >>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and
>> nail
>> >>>>>> them.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> Our tests now run in parallel with other test
>> suites
>> >>>>>> where
>> >>>>>> > > > previous
>> >>>>>> > > > > > we
>> >>>>>> > > > > > > >>> ran alone. You can see this sometimes when our
>> zombie
>> >>>>>> > detector
>> >>>>>> > > > > > reports
>> >>>>>> > > > > > > >>> tests from another project altogether as lingerers
>> >>>>>> (To be
>> >>>>>> > > fixed).
>> >>>>>> > > > > > > Some of
>> >>>>>> > > > > > > >>> our tests are failing because a concurrent hbase
>> run
>> >>>>>> is
>> >>>>>> > undoing
>> >>>>>> > > > > > > classes and
>> >>>>>> > > > > > > >>> data from under it. Also, lets fix.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them
>> to
>> >>>>>> > complete.
>> >>>>>> > > > > Many
>> >>>>>> > > > > > > >>> are heavy-duty integration tests starting up
>> multiple
>> >>>>>> > clusters
>> >>>>>> > > > and
>> >>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they
>> >>>>>> pass at
>> >>>>>> > all.
>> >>>>>> > > > > > > Usually
>> >>>>>> > > > > > > >>> integration tests have been cast as unit tests
>> >>>>>> because there
>> >>>>>> > > was
>> >>>>>> > > > no
>> >>>>>> > > > > > > where
>> >>>>>> > > > > > > >>> else for them to get an airing. We have the
>> hbase-it
>> >>>>>> suite
>> >>>>>> > now
>> >>>>>> > > > > which
>> >>>>>> > > > > > > would
>> >>>>>> > > > > > > >>> be a more apt place but until these are run on a
>> >>>>>> regular
>> >>>>>> > basis
>> >>>>>> > > in
>> >>>>>> > > > > > > public
>> >>>>>> > > > > > > >>> for all to see, the fat integration tests disguised
>> >>>>>> as unit
>> >>>>>> > > tests
>> >>>>>> > > > > > will
>> >>>>>> > > > > > > >>> remain. A review of our current unit tests weeding
>> >>>>>> the old
>> >>>>>> > > cruft
>> >>>>>> > > > > and
>> >>>>>> > > > > > > the
>> >>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
>> >>>>>> undertaking
>> >>>>>> > if
>> >>>>>> > > > > > > someone is
>> >>>>>> > > > > > > >>> looking to contribute.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> Alex Newman has been working on making our tests
>> work
>> >>>>>> up on
>> >>>>>> > > > travis
>> >>>>>> > > > > > and
>> >>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes
>> >>>>>> end-to-end. He
>> >>>>>> > also
>> >>>>>> > > > > added
>> >>>>>> > > > > > in
>> >>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
>> >>>>>> mapreduce --
>> >>>>>> > > > > alongside
>> >>>>>> > > > > > > our
>> >>>>>> > > > > > > >>> old "sizing" categorizations of small/medium/large.
>> >>>>>> His
>> >>>>>> > > thinking
>> >>>>>> > > > > is
>> >>>>>> > > > > > > that
>> >>>>>> > > > > > > >>> we can run these categorizations in parallel so we
>> >>>>>> could run
>> >>>>>> > > the
>> >>>>>> > > > > > total
>> >>>>>> > > > > > > >>> suite in about the time of the longest test, say
>> >>>>>> > 20-30minutes?
>> >>>>>> > > > We
>> >>>>>> > > > > > > could
>> >>>>>> > > > > > > >>> even change Apache to run them this way.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> FYI,
>> >>>>>> > > > > > > >>> St.Ack
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >
>> >>>>>> > > > > > >
>> >>>>>> > > > > >
>> >>>>>> > > > > >
>> >>>>>> > > > > >
>> >>>>>> > > > > > --
>> >>>>>> > > > > > Sean
>> >>>>>> > > > > >
>> >>>>>> > > > >
>> >>>>>> > > >
>> >>>>>> > > >
>> >>>>>> > > >
>> >>>>>> > > > --
>> >>>>>> > > > Best regards,
>> >>>>>> > > >
>> >>>>>> > > > - Andy
>> >>>>>> > > >
>> >>>>>> > > > Problems worthy of attack prove their worth by hitting back.
>> -
>> >>>>>> Piet
>> >>>>>> > Hein
>> >>>>>> > > > (via Tom White)
>> >>>>>> > > >
>> >>>>>> > >
>> >>>>>> >
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Sean
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Sean
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Sean
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Sean
>> >>
>> >
>> >
>> >
>> > --
>> > Sean
>> >
>>
>>
>>
>> --
>> Sean
>>
>
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Enis Söztutar <en...@gmail.com>.
hadoopqa is acting weird for the last couple of days. It does not report
back the results. One example is:
https://builds.apache.org/job/PreCommit-HBASE-Build/14475/console.
I've excluded ubuntu3 from the list because it does not have protoc-2.5.
On Mon, Jun 15, 2015 at 9:55 PM, Sean Busbey <bu...@cloudera.com> wrote:
> Added new HBase-1.3 build or branch-1 and changed the existing 1.2 builds
> to use branch-1.2. (ref HBASE-13912)
>
> On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com> wrote:
>
> > I've add a new build test to run the ITs under sunny day conditions using
> > minicluster for both java 7 and java 8 on the 1.2 line.
> >
> > https://builds.apache.org/job/HBase-1.2-IT/
> >
> > I haven't turned on notifications yet, because BulkIngest and TestIngest
> > are read. details on HBASE-13750
> >
> > On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> >> sigh. that should have been "is now a matrix build".
> >>
> >> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >>
> >>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 in
> >>> parallel.
> >>>
> >>> I saved the old job for now and left it disabled[2].
> >>>
> >>>
> >>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
> >>> [2]:
> >>>
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
> >>>
> >>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com>
> >>> wrote:
> >>>
> >>>> Sorry for the noise. I also updated the checkstyle step on HBase-TRUNK
> >>>>
> >>>> from
> >>>> mvn checkstyle:checkstyle
> >>>> to
> >>>> mvn -DskipTests package checkstyle:checkstyle
> >>>>
> >>>> to deal with the same issue. Looks all clear now.
> >>>>
> >>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com>
> >>>> wrote:
> >>>>
> >>>>> Updated the following builds:
> >>>>>
> >>>>> * HBase-TRUNK
> >>>>>
> >>>>> moved from
> >>>>>
> >>>>> mvn clean compile findbugs:findbugs
> >>>>>
> >>>>> to
> >>>>>
> >>>>> mvn clean -DskipTests package findbugs:findbugs
> >>>>>
> >>>>> To work around the known issue where we can't do a bootstrap compile
> >>>>> without previous install or remote SNAPSHOT artifacts. (recently
> triggered
> >>>>> by the addition of the procedure module on master)
> >>>>>
> >>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
> >>>>>
> >>>>>> Make findbugs and checkstyle plugins always run.
> >>>>>> The default behavior only publishes result for stable builds, but
> >>>>>> master is
> >>>>>> not always stable and sometimes we introduce new warnings in
> unstable
> >>>>>> builds(see builds 6310-6314).
> >>>>>> And findbugs and checkstyle will not fail unless we abort the
> building
> >>>>>> process I think, so it is safe to turn it on every time.
> >>>>>>
> >>>>>> Thanks.
> >>>>>>
> >>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
> >>>>>>
> >>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
> >>>>>> >
> >>>>>> > Cheers
> >>>>>> >
> >>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com>
> wrote:
> >>>>>> >
> >>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
> >>>>>> findbugs,
> >>>>>> > > checkstyle and jacoco coverage report.
> >>>>>> > > The config has been tested on
> >>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30
> >>>>>> times.
> >>>>>> > Not
> >>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
> >>>>>> overhead
> >>>>>> > of
> >>>>>> > > collecting information for code coverage).
> >>>>>> > > Thanks.
> >>>>>> > >
> >>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
> >>>>>> > >
> >>>>>> > > > +1, thanks a lot for improving our build hygiene.
> >>>>>> > > >
> >>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <
> >>>>>> enis.soz@gmail.com>
> >>>>>> > > wrote:
> >>>>>> > > >
> >>>>>> > > > > Thanks Sean. This is great.
> >>>>>> > > > >
> >>>>>> > > > > Enis
> >>>>>> > > > >
> >>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <
> >>>>>> busbey@cloudera.com>
> >>>>>> > > > wrote:
> >>>>>> > > > >
> >>>>>> > > > > > FYI I just finished chasing down the breakage for mvn site
> >>>>>> on all
> >>>>>> > > patch
> >>>>>> > > > > > builds.
> >>>>>> > > > > >
> >>>>>> > > > > > HBASE-13191 consolidates the few places in the test-patch
> >>>>>> code
> >>>>>> > where
> >>>>>> > > we
> >>>>>> > > > > > hard coded MAVEN_OPTS.
> >>>>>> > > > > >
> >>>>>> > > > > > If you look at the PreCommit job now, we use the "set
> >>>>>> environment
> >>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything
> else
> >>>>>> > respects
> >>>>>> > > > > that
> >>>>>> > > > > > setting.
> >>>>>> > > > > >
> >>>>>> > > > > > I set the initial value to be a combination of the memory
> >>>>>> > limitations
> >>>>>> > > > > we've
> >>>>>> > > > > > been actually running with (the ~6G was getting ignored)
> >>>>>> and the
> >>>>>> > > > permgen
> >>>>>> > > > > > needed for site.
> >>>>>> > > > > >
> >>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
> >>>>>> > > > > >
> >>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <stack@duboce.net
> >
> >>>>>> wrote:
> >>>>>> > > > > >
> >>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
> >>>>>> patch-build
> >>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to
> >>>>>> be the
> >>>>>> > > same
> >>>>>> > > > as
> >>>>>> > > > > > > those of trunk build too, setting
> >>>>>> MAVEN_OPTS="-Xmx6100m"... it
> >>>>>> > had
> >>>>>> > > > been
> >>>>>> > > > > > > 3000.
> >>>>>> > > > > > >
> >>>>>> > > > > > > Yours,
> >>>>>> > > > > > > St.Ack
> >>>>>> > > > > > >
> >>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <
> stack@duboce.net>
> >>>>>> wrote:
> >>>>>> > > > > > >
> >>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds and
> >>>>>> or last
> >>>>>> > 7
> >>>>>> > > > > days,
> >>>>>> > > > > > > > whichever comes first.
> >>>>>> > > > > > > > St.Ack
> >>>>>> > > > > > > >
> >>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <
> stack@duboce.net
> >>>>>> >
> >>>>>> > wrote:
> >>>>>> > > > > > > >
> >>>>>> > > > > > > >> Branch-1 and master have stabilized and now run
> mostly
> >>>>>> blue
> >>>>>> > > (give
> >>>>>> > > > or
> >>>>>> > > > > > > take
> >>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue
> branch-1
> >>>>>> has
> >>>>>> > > helped
> >>>>>> > > > us
> >>>>>> > > > > > > >> identify at least one destabilizing commit in the
> last
> >>>>>> few
> >>>>>> > days,
> >>>>>> > > > > maybe
> >>>>>> > > > > > > two;
> >>>>>> > > > > > > >> this is as it should be (smile).
> >>>>>> > > > > > > >>
> >>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch,
> make
> >>>>>> sure
> >>>>>> > > > > subsequent
> >>>>>> > > > > > > >> builds stay blue. You can subscribe to
> >>>>>> > builds@hbase.apache.org
> >>>>>> > > to
> >>>>>> > > > > get
> >>>>>> > > > > > > >> notice of failures if not already subscribed.
> >>>>>> > > > > > > >>
> >>>>>> > > > > > > >> Thanks,
> >>>>>> > > > > > > >> St.Ack
> >>>>>> > > > > > > >>
> >>>>>> > > > > > > >> 1.
> >>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> >>>>>> > > > > > > >> 2.
> >>>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> >>>>>> > > > > > > >>
> >>>>>> > > > > > > >>
> >>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <
> >>>>>> stack@duboce.net>
> >>>>>> > > wrote:
> >>>>>> > > > > > > >>
> >>>>>> > > > > > > >>> A few notes on testing.
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>> Too long to read, infra is more capable now and
> after
> >>>>>> some
> >>>>>> > > work,
> >>>>>> > > > we
> >>>>>> > > > > > are
> >>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets
> >>>>>> try and
> >>>>>> > > keep
> >>>>>> > > > it
> >>>>>> > > > > > > this
> >>>>>> > > > > > > >>> way going forward.
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>> A recent spurt of test fixing combined with more
> >>>>>> capable
> >>>>>> > > hardware
> >>>>>> > > > > > seems
> >>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
> >>>>>> passing
> >>>>>> > now
> >>>>>> > > on
> >>>>>> > > > > > > branch-1
> >>>>>> > > > > > > >>> and master. Lets try and keep it this way and start
> >>>>>> to trust
> >>>>>> > > our
> >>>>>> > > > > > test
> >>>>>> > > > > > > runs
> >>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and
> nail
> >>>>>> them.
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>> Our tests now run in parallel with other test suites
> >>>>>> where
> >>>>>> > > > previous
> >>>>>> > > > > > we
> >>>>>> > > > > > > >>> ran alone. You can see this sometimes when our
> zombie
> >>>>>> > detector
> >>>>>> > > > > > reports
> >>>>>> > > > > > > >>> tests from another project altogether as lingerers
> >>>>>> (To be
> >>>>>> > > fixed).
> >>>>>> > > > > > > Some of
> >>>>>> > > > > > > >>> our tests are failing because a concurrent hbase run
> >>>>>> is
> >>>>>> > undoing
> >>>>>> > > > > > > classes and
> >>>>>> > > > > > > >>> data from under it. Also, lets fix.
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them
> to
> >>>>>> > complete.
> >>>>>> > > > > Many
> >>>>>> > > > > > > >>> are heavy-duty integration tests starting up
> multiple
> >>>>>> > clusters
> >>>>>> > > > and
> >>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they
> >>>>>> pass at
> >>>>>> > all.
> >>>>>> > > > > > > Usually
> >>>>>> > > > > > > >>> integration tests have been cast as unit tests
> >>>>>> because there
> >>>>>> > > was
> >>>>>> > > > no
> >>>>>> > > > > > > where
> >>>>>> > > > > > > >>> else for them to get an airing. We have the
> hbase-it
> >>>>>> suite
> >>>>>> > now
> >>>>>> > > > > which
> >>>>>> > > > > > > would
> >>>>>> > > > > > > >>> be a more apt place but until these are run on a
> >>>>>> regular
> >>>>>> > basis
> >>>>>> > > in
> >>>>>> > > > > > > public
> >>>>>> > > > > > > >>> for all to see, the fat integration tests disguised
> >>>>>> as unit
> >>>>>> > > tests
> >>>>>> > > > > > will
> >>>>>> > > > > > > >>> remain. A review of our current unit tests weeding
> >>>>>> the old
> >>>>>> > > cruft
> >>>>>> > > > > and
> >>>>>> > > > > > > the
> >>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
> >>>>>> undertaking
> >>>>>> > if
> >>>>>> > > > > > > someone is
> >>>>>> > > > > > > >>> looking to contribute.
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>> Alex Newman has been working on making our tests
> work
> >>>>>> up on
> >>>>>> > > > travis
> >>>>>> > > > > > and
> >>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes
> >>>>>> end-to-end. He
> >>>>>> > also
> >>>>>> > > > > added
> >>>>>> > > > > > in
> >>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
> >>>>>> mapreduce --
> >>>>>> > > > > alongside
> >>>>>> > > > > > > our
> >>>>>> > > > > > > >>> old "sizing" categorizations of small/medium/large.
> >>>>>> His
> >>>>>> > > thinking
> >>>>>> > > > > is
> >>>>>> > > > > > > that
> >>>>>> > > > > > > >>> we can run these categorizations in parallel so we
> >>>>>> could run
> >>>>>> > > the
> >>>>>> > > > > > total
> >>>>>> > > > > > > >>> suite in about the time of the longest test, say
> >>>>>> > 20-30minutes?
> >>>>>> > > > We
> >>>>>> > > > > > > could
> >>>>>> > > > > > > >>> even change Apache to run them this way.
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>> FYI,
> >>>>>> > > > > > > >>> St.Ack
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>>
> >>>>>> > > > > > > >>
> >>>>>> > > > > > > >
> >>>>>> > > > > > >
> >>>>>> > > > > >
> >>>>>> > > > > >
> >>>>>> > > > > >
> >>>>>> > > > > > --
> >>>>>> > > > > > Sean
> >>>>>> > > > > >
> >>>>>> > > > >
> >>>>>> > > >
> >>>>>> > > >
> >>>>>> > > >
> >>>>>> > > > --
> >>>>>> > > > Best regards,
> >>>>>> > > >
> >>>>>> > > > - Andy
> >>>>>> > > >
> >>>>>> > > > Problems worthy of attack prove their worth by hitting back. -
> >>>>>> Piet
> >>>>>> > Hein
> >>>>>> > > > (via Tom White)
> >>>>>> > > >
> >>>>>> > >
> >>>>>> >
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Sean
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Sean
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Sean
> >>>
> >>
> >>
> >>
> >> --
> >> Sean
> >>
> >
> >
> >
> > --
> > Sean
> >
>
>
>
> --
> Sean
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by stack <sa...@gmail.com>.
Thank you Sean (and Andrew)
St.Ack
On Jan 22, 2016 10:12 PM, "Sean Busbey" <bu...@apache.org> wrote:
> Andrew in infra made us a label that covers all the hosts save H2.
> I've updated all the nightly builds to use it.
>
> (specifying a label expression as we do in precommit doesn't work on
> matrix builds because the & and | from the expression end up in the
> filesystem path)
>
> On Fri, Jan 22, 2016 at 3:08 PM, Sean Busbey <bu...@cloudera.com> wrote:
> > we should probably ensure the earlier branch builds also exclude H2.
> >
> > I'll leave myself a note to look at it this evening. If anyone gets to it
> > before then, please update here.
> >
> > On Fri, Jan 22, 2016 at 1:33 PM, Stack <st...@duboce.net> wrote:
> >
> >> Related to the below, I just changed the trunk matrix build job to
> exclude
> >> H2 from the build roster (with Sean's help); it seems to be responsible
> for
> >> this failure type -- *Caused by: java.lang.IndexOutOfBoundsException:
> >> Index: 0, Size: 0* -- in particular. Here is recent example:
> >>
> >>
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/652/jdk=latest1.7,label=Hadoop/console
> >>
> >> Lets see if it helps.
> >>
> >> While I have your attention, see the nice checkstyle and findbug graph
> >> trajectories here:
> >> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/
> >>
> >> St.Ack
> >>
> >>
> >> On Tue, Jan 19, 2016 at 7:02 PM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >>
> >> > On Tue, Jan 19, 2016 at 11:48 AM, Stack <st...@duboce.net> wrote:
> >> >
> >> > > On Tue, Jan 19, 2016 at 5:46 AM, Sean Busbey <bu...@apache.org>
> >> wrote:
> >> > >
> >> > > > We could start forcing a clean repository on every build (though
> this
> >> > > > seems heavy handed).
> >> > > >
> >> > > > IIRC, we ran into this ages ago and it was one particular
> dependency.
> >> > > > Presuming we can track down what that was, we could add some
> >> pre-build
> >> > > > work that verifies a known good copy of that dependency is
> present.
> >> > > >
> >> > > > For now, I've blacklisted the H2 build host again, since that's
> the
> >> > > > only host this has been happening on. We added it back last week
> so
> >> > > > that Jon could test a fix from infra.
> >> > > >
> >> > > >
> >> > > Thanks Sean.
> >> > >
> >> > > Yeah, clean repo each time would be OTT.
> >> > >
> >> > > If you have pointers on how to find the bad dependency, I'll dig.
> >> > >
> >> > > Of course, all builds fine locally.
> >> > >
> >> > > St.Ack
> >> > >
> >> > >
> >> > >
> >> > Relevant bits from our old precommit build (lucky we kept it! ;) )
> >> >
> >> >
> >> > ----
> >> > ...
> >> > # holding place for local repo
> >> > if [ -d "${WORKSPACE}/maven_repo" ]; then
> >> > echo "removing hbase artifacts from prior runs."
> >> > rm -rf "${WORKSPACE}/maven_repo/org/apache/hbase"
> >> > echo "removing javax.inject in case we got a bad dependency"
> >> > rm -rf "${WORKSPACE}/maven_repo/javax/inject"
> >> > # uncomment to list entire contents of repo
> >> > # find "${WORKSPACE}/maven_repo"
> >> > else
> >> > mkdir "${WORKSPACE}/maven_repo"
> >> > fi
> >> >
> >> > ...
> >> >
> >> > if [ "$?" -ne "0" ]; then
> >> > echo "test patch failed, checking javax.inject dependency"
> >> > find "${WORKSPACE}/maven_repo/javax/inject"
> >> > cat
> >> >
> "${WORKSPACE}/maven_repo/javax/inject/javax.inject/1/javax.inject-1.pom"
> >> > exit 1
> >> > fi
> >> > ---
> >> >
> >> > So it looks like javax:inject was the culprit before. A first step
> would
> >> be
> >> > to remove it from our local repos before running; that'll require
> looking
> >> > up how Yetus names the branch-specific maven repos. A better step
> would
> >> be
> >> > to manually install it from a known good location so we can avoid
> >> whatever
> >> > nonsense is happening on H2.
> >> >
> >>
> >
> >
> >
> > --
> > Sean
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@apache.org>.
Andrew in infra made us a label that covers all the hosts save H2.
I've updated all the nightly builds to use it.
(specifying a label expression as we do in precommit doesn't work on
matrix builds because the & and | from the expression end up in the
filesystem path)
On Fri, Jan 22, 2016 at 3:08 PM, Sean Busbey <bu...@cloudera.com> wrote:
> we should probably ensure the earlier branch builds also exclude H2.
>
> I'll leave myself a note to look at it this evening. If anyone gets to it
> before then, please update here.
>
> On Fri, Jan 22, 2016 at 1:33 PM, Stack <st...@duboce.net> wrote:
>
>> Related to the below, I just changed the trunk matrix build job to exclude
>> H2 from the build roster (with Sean's help); it seems to be responsible for
>> this failure type -- *Caused by: java.lang.IndexOutOfBoundsException:
>> Index: 0, Size: 0* -- in particular. Here is recent example:
>>
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/652/jdk=latest1.7,label=Hadoop/console
>>
>> Lets see if it helps.
>>
>> While I have your attention, see the nice checkstyle and findbug graph
>> trajectories here:
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/
>>
>> St.Ack
>>
>>
>> On Tue, Jan 19, 2016 at 7:02 PM, Sean Busbey <bu...@cloudera.com> wrote:
>>
>> > On Tue, Jan 19, 2016 at 11:48 AM, Stack <st...@duboce.net> wrote:
>> >
>> > > On Tue, Jan 19, 2016 at 5:46 AM, Sean Busbey <bu...@apache.org>
>> wrote:
>> > >
>> > > > We could start forcing a clean repository on every build (though this
>> > > > seems heavy handed).
>> > > >
>> > > > IIRC, we ran into this ages ago and it was one particular dependency.
>> > > > Presuming we can track down what that was, we could add some
>> pre-build
>> > > > work that verifies a known good copy of that dependency is present.
>> > > >
>> > > > For now, I've blacklisted the H2 build host again, since that's the
>> > > > only host this has been happening on. We added it back last week so
>> > > > that Jon could test a fix from infra.
>> > > >
>> > > >
>> > > Thanks Sean.
>> > >
>> > > Yeah, clean repo each time would be OTT.
>> > >
>> > > If you have pointers on how to find the bad dependency, I'll dig.
>> > >
>> > > Of course, all builds fine locally.
>> > >
>> > > St.Ack
>> > >
>> > >
>> > >
>> > Relevant bits from our old precommit build (lucky we kept it! ;) )
>> >
>> >
>> > ----
>> > ...
>> > # holding place for local repo
>> > if [ -d "${WORKSPACE}/maven_repo" ]; then
>> > echo "removing hbase artifacts from prior runs."
>> > rm -rf "${WORKSPACE}/maven_repo/org/apache/hbase"
>> > echo "removing javax.inject in case we got a bad dependency"
>> > rm -rf "${WORKSPACE}/maven_repo/javax/inject"
>> > # uncomment to list entire contents of repo
>> > # find "${WORKSPACE}/maven_repo"
>> > else
>> > mkdir "${WORKSPACE}/maven_repo"
>> > fi
>> >
>> > ...
>> >
>> > if [ "$?" -ne "0" ]; then
>> > echo "test patch failed, checking javax.inject dependency"
>> > find "${WORKSPACE}/maven_repo/javax/inject"
>> > cat
>> > "${WORKSPACE}/maven_repo/javax/inject/javax.inject/1/javax.inject-1.pom"
>> > exit 1
>> > fi
>> > ---
>> >
>> > So it looks like javax:inject was the culprit before. A first step would
>> be
>> > to remove it from our local repos before running; that'll require looking
>> > up how Yetus names the branch-specific maven repos. A better step would
>> be
>> > to manually install it from a known good location so we can avoid
>> whatever
>> > nonsense is happening on H2.
>> >
>>
>
>
>
> --
> Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
we should probably ensure the earlier branch builds also exclude H2.
I'll leave myself a note to look at it this evening. If anyone gets to it
before then, please update here.
On Fri, Jan 22, 2016 at 1:33 PM, Stack <st...@duboce.net> wrote:
> Related to the below, I just changed the trunk matrix build job to exclude
> H2 from the build roster (with Sean's help); it seems to be responsible for
> this failure type -- *Caused by: java.lang.IndexOutOfBoundsException:
> Index: 0, Size: 0* -- in particular. Here is recent example:
>
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/652/jdk=latest1.7,label=Hadoop/console
>
> Lets see if it helps.
>
> While I have your attention, see the nice checkstyle and findbug graph
> trajectories here:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/
>
> St.Ack
>
>
> On Tue, Jan 19, 2016 at 7:02 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
> > On Tue, Jan 19, 2016 at 11:48 AM, Stack <st...@duboce.net> wrote:
> >
> > > On Tue, Jan 19, 2016 at 5:46 AM, Sean Busbey <bu...@apache.org>
> wrote:
> > >
> > > > We could start forcing a clean repository on every build (though this
> > > > seems heavy handed).
> > > >
> > > > IIRC, we ran into this ages ago and it was one particular dependency.
> > > > Presuming we can track down what that was, we could add some
> pre-build
> > > > work that verifies a known good copy of that dependency is present.
> > > >
> > > > For now, I've blacklisted the H2 build host again, since that's the
> > > > only host this has been happening on. We added it back last week so
> > > > that Jon could test a fix from infra.
> > > >
> > > >
> > > Thanks Sean.
> > >
> > > Yeah, clean repo each time would be OTT.
> > >
> > > If you have pointers on how to find the bad dependency, I'll dig.
> > >
> > > Of course, all builds fine locally.
> > >
> > > St.Ack
> > >
> > >
> > >
> > Relevant bits from our old precommit build (lucky we kept it! ;) )
> >
> >
> > ----
> > ...
> > # holding place for local repo
> > if [ -d "${WORKSPACE}/maven_repo" ]; then
> > echo "removing hbase artifacts from prior runs."
> > rm -rf "${WORKSPACE}/maven_repo/org/apache/hbase"
> > echo "removing javax.inject in case we got a bad dependency"
> > rm -rf "${WORKSPACE}/maven_repo/javax/inject"
> > # uncomment to list entire contents of repo
> > # find "${WORKSPACE}/maven_repo"
> > else
> > mkdir "${WORKSPACE}/maven_repo"
> > fi
> >
> > ...
> >
> > if [ "$?" -ne "0" ]; then
> > echo "test patch failed, checking javax.inject dependency"
> > find "${WORKSPACE}/maven_repo/javax/inject"
> > cat
> > "${WORKSPACE}/maven_repo/javax/inject/javax.inject/1/javax.inject-1.pom"
> > exit 1
> > fi
> > ---
> >
> > So it looks like javax:inject was the culprit before. A first step would
> be
> > to remove it from our local repos before running; that'll require looking
> > up how Yetus names the branch-specific maven repos. A better step would
> be
> > to manually install it from a known good location so we can avoid
> whatever
> > nonsense is happening on H2.
> >
>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
Related to the below, I just changed the trunk matrix build job to exclude
H2 from the build roster (with Sean's help); it seems to be responsible for
this failure type -- *Caused by: java.lang.IndexOutOfBoundsException:
Index: 0, Size: 0* -- in particular. Here is recent example:
https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/652/jdk=latest1.7,label=Hadoop/console
Lets see if it helps.
While I have your attention, see the nice checkstyle and findbug graph
trajectories here:
https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/
St.Ack
On Tue, Jan 19, 2016 at 7:02 PM, Sean Busbey <bu...@cloudera.com> wrote:
> On Tue, Jan 19, 2016 at 11:48 AM, Stack <st...@duboce.net> wrote:
>
> > On Tue, Jan 19, 2016 at 5:46 AM, Sean Busbey <bu...@apache.org> wrote:
> >
> > > We could start forcing a clean repository on every build (though this
> > > seems heavy handed).
> > >
> > > IIRC, we ran into this ages ago and it was one particular dependency.
> > > Presuming we can track down what that was, we could add some pre-build
> > > work that verifies a known good copy of that dependency is present.
> > >
> > > For now, I've blacklisted the H2 build host again, since that's the
> > > only host this has been happening on. We added it back last week so
> > > that Jon could test a fix from infra.
> > >
> > >
> > Thanks Sean.
> >
> > Yeah, clean repo each time would be OTT.
> >
> > If you have pointers on how to find the bad dependency, I'll dig.
> >
> > Of course, all builds fine locally.
> >
> > St.Ack
> >
> >
> >
> Relevant bits from our old precommit build (lucky we kept it! ;) )
>
>
> ----
> ...
> # holding place for local repo
> if [ -d "${WORKSPACE}/maven_repo" ]; then
> echo "removing hbase artifacts from prior runs."
> rm -rf "${WORKSPACE}/maven_repo/org/apache/hbase"
> echo "removing javax.inject in case we got a bad dependency"
> rm -rf "${WORKSPACE}/maven_repo/javax/inject"
> # uncomment to list entire contents of repo
> # find "${WORKSPACE}/maven_repo"
> else
> mkdir "${WORKSPACE}/maven_repo"
> fi
>
> ...
>
> if [ "$?" -ne "0" ]; then
> echo "test patch failed, checking javax.inject dependency"
> find "${WORKSPACE}/maven_repo/javax/inject"
> cat
> "${WORKSPACE}/maven_repo/javax/inject/javax.inject/1/javax.inject-1.pom"
> exit 1
> fi
> ---
>
> So it looks like javax:inject was the culprit before. A first step would be
> to remove it from our local repos before running; that'll require looking
> up how Yetus names the branch-specific maven repos. A better step would be
> to manually install it from a known good location so we can avoid whatever
> nonsense is happening on H2.
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
On Tue, Jan 19, 2016 at 11:48 AM, Stack <st...@duboce.net> wrote:
> On Tue, Jan 19, 2016 at 5:46 AM, Sean Busbey <bu...@apache.org> wrote:
>
> > We could start forcing a clean repository on every build (though this
> > seems heavy handed).
> >
> > IIRC, we ran into this ages ago and it was one particular dependency.
> > Presuming we can track down what that was, we could add some pre-build
> > work that verifies a known good copy of that dependency is present.
> >
> > For now, I've blacklisted the H2 build host again, since that's the
> > only host this has been happening on. We added it back last week so
> > that Jon could test a fix from infra.
> >
> >
> Thanks Sean.
>
> Yeah, clean repo each time would be OTT.
>
> If you have pointers on how to find the bad dependency, I'll dig.
>
> Of course, all builds fine locally.
>
> St.Ack
>
>
>
Relevant bits from our old precommit build (lucky we kept it! ;) )
----
...
# holding place for local repo
if [ -d "${WORKSPACE}/maven_repo" ]; then
echo "removing hbase artifacts from prior runs."
rm -rf "${WORKSPACE}/maven_repo/org/apache/hbase"
echo "removing javax.inject in case we got a bad dependency"
rm -rf "${WORKSPACE}/maven_repo/javax/inject"
# uncomment to list entire contents of repo
# find "${WORKSPACE}/maven_repo"
else
mkdir "${WORKSPACE}/maven_repo"
fi
...
if [ "$?" -ne "0" ]; then
echo "test patch failed, checking javax.inject dependency"
find "${WORKSPACE}/maven_repo/javax/inject"
cat
"${WORKSPACE}/maven_repo/javax/inject/javax.inject/1/javax.inject-1.pom"
exit 1
fi
---
So it looks like javax:inject was the culprit before. A first step would be
to remove it from our local repos before running; that'll require looking
up how Yetus names the branch-specific maven repos. A better step would be
to manually install it from a known good location so we can avoid whatever
nonsense is happening on H2.
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
On Tue, Jan 19, 2016 at 5:46 AM, Sean Busbey <bu...@apache.org> wrote:
> We could start forcing a clean repository on every build (though this
> seems heavy handed).
>
> IIRC, we ran into this ages ago and it was one particular dependency.
> Presuming we can track down what that was, we could add some pre-build
> work that verifies a known good copy of that dependency is present.
>
> For now, I've blacklisted the H2 build host again, since that's the
> only host this has been happening on. We added it back last week so
> that Jon could test a fix from infra.
>
>
Thanks Sean.
Yeah, clean repo each time would be OTT.
If you have pointers on how to find the bad dependency, I'll dig.
Of course, all builds fine locally.
St.Ack
> -Sean
>
> On Mon, Jan 18, 2016 at 10:55 PM, Stack <st...@duboce.net> wrote:
> > Anyone know what the refresh timeout is for the below? We seem to be in a
> > phase where we have a bad pom and the hadoop test builds are failing. Can
> > we force refresh of the local repository by doing something like a custom
> > build run?
> >
> > Thanks,
> > St.Ack
> > P.S. Here is what I am talking about:
> >
> https://builds.apache.org/job/PreCommit-HBASE-Build/178/artifact/patchprocess/patch-javac-2.4.0.txt
> > which is from this run today:
> > https://issues.apache.org/jira/browse/HBASE-15086 Thanks
> >
> >
> > On Thu, Nov 5, 2015 at 10:42 AM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> >> If Maven has trouble grabbing a pom but not an artifact, it'll
> >> substitute in a placeholder pom that doesn't have e.g. license
> >> information. That can result in a local repo that fails this way until
> >> the refresh timeout hits for grabbing a pom again.
> >>
> >> On Wed, Nov 4, 2015 at 5:33 PM, Stack <st...@duboce.net> wrote:
> >> > Thanks Andrew. Weird is that it is sporadic. Will keep an eye on it.
> >> > St.Ack
> >> >
> >> > On Wed, Nov 4, 2015 at 8:14 AM, Andrew Purtell <
> andrew.purtell@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
> >> >> java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
> >> >>
> >> >> This means a Velocity macro for building LICENSE info about a
> component
> >> >> has failed because necessary information is missing in the Maven
> model.
> >> >> When I have seen this it has been due to a change in dependencies
> >> without
> >> >> necessary updates to supplemental-models.xml. Running the LICENSE and
> >> >> NOTICE aggregations in debug mode will print out clues as to what
> >> >> specifically is missing. See defines in the POMs for doing so. (I'm
> not
> >> at
> >> >> the computer yet so can't pull up the specifics.)
> >> >>
> >> >>
> >> >> > On Nov 4, 2015, at 6:23 AM, Stack <st...@duboce.net> wrote:
> >> >> >
> >> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
> >> >> > java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
> >> >>
> >>
> >>
> >>
> >> --
> >> Sean
> >>
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@apache.org>.
We could start forcing a clean repository on every build (though this
seems heavy handed).
IIRC, we ran into this ages ago and it was one particular dependency.
Presuming we can track down what that was, we could add some pre-build
work that verifies a known good copy of that dependency is present.
For now, I've blacklisted the H2 build host again, since that's the
only host this has been happening on. We added it back last week so
that Jon could test a fix from infra.
-Sean
On Mon, Jan 18, 2016 at 10:55 PM, Stack <st...@duboce.net> wrote:
> Anyone know what the refresh timeout is for the below? We seem to be in a
> phase where we have a bad pom and the hadoop test builds are failing. Can
> we force refresh of the local repository by doing something like a custom
> build run?
>
> Thanks,
> St.Ack
> P.S. Here is what I am talking about:
> https://builds.apache.org/job/PreCommit-HBASE-Build/178/artifact/patchprocess/patch-javac-2.4.0.txt
> which is from this run today:
> https://issues.apache.org/jira/browse/HBASE-15086 Thanks
>
>
> On Thu, Nov 5, 2015 at 10:42 AM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> If Maven has trouble grabbing a pom but not an artifact, it'll
>> substitute in a placeholder pom that doesn't have e.g. license
>> information. That can result in a local repo that fails this way until
>> the refresh timeout hits for grabbing a pom again.
>>
>> On Wed, Nov 4, 2015 at 5:33 PM, Stack <st...@duboce.net> wrote:
>> > Thanks Andrew. Weird is that it is sporadic. Will keep an eye on it.
>> > St.Ack
>> >
>> > On Wed, Nov 4, 2015 at 8:14 AM, Andrew Purtell <andrew.purtell@gmail.com
>> >
>> > wrote:
>> >
>> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
>> >> java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
>> >>
>> >> This means a Velocity macro for building LICENSE info about a component
>> >> has failed because necessary information is missing in the Maven model.
>> >> When I have seen this it has been due to a change in dependencies
>> without
>> >> necessary updates to supplemental-models.xml. Running the LICENSE and
>> >> NOTICE aggregations in debug mode will print out clues as to what
>> >> specifically is missing. See defines in the POMs for doing so. (I'm not
>> at
>> >> the computer yet so can't pull up the specifics.)
>> >>
>> >>
>> >> > On Nov 4, 2015, at 6:23 AM, Stack <st...@duboce.net> wrote:
>> >> >
>> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
>> >> > java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
>> >>
>>
>>
>>
>> --
>> Sean
>>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
Anyone know what the refresh timeout is for the below? We seem to be in a
phase where we have a bad pom and the hadoop test builds are failing. Can
we force refresh of the local repository by doing something like a custom
build run?
Thanks,
St.Ack
P.S. Here is what I am talking about:
https://builds.apache.org/job/PreCommit-HBASE-Build/178/artifact/patchprocess/patch-javac-2.4.0.txt
which is from this run today:
https://issues.apache.org/jira/browse/HBASE-15086 Thanks
On Thu, Nov 5, 2015 at 10:42 AM, Sean Busbey <bu...@cloudera.com> wrote:
> If Maven has trouble grabbing a pom but not an artifact, it'll
> substitute in a placeholder pom that doesn't have e.g. license
> information. That can result in a local repo that fails this way until
> the refresh timeout hits for grabbing a pom again.
>
> On Wed, Nov 4, 2015 at 5:33 PM, Stack <st...@duboce.net> wrote:
> > Thanks Andrew. Weird is that it is sporadic. Will keep an eye on it.
> > St.Ack
> >
> > On Wed, Nov 4, 2015 at 8:14 AM, Andrew Purtell <andrew.purtell@gmail.com
> >
> > wrote:
> >
> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
> >> java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
> >>
> >> This means a Velocity macro for building LICENSE info about a component
> >> has failed because necessary information is missing in the Maven model.
> >> When I have seen this it has been due to a change in dependencies
> without
> >> necessary updates to supplemental-models.xml. Running the LICENSE and
> >> NOTICE aggregations in debug mode will print out clues as to what
> >> specifically is missing. See defines in the POMs for doing so. (I'm not
> at
> >> the computer yet so can't pull up the specifics.)
> >>
> >>
> >> > On Nov 4, 2015, at 6:23 AM, Stack <st...@duboce.net> wrote:
> >> >
> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
> >> > java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
> >>
>
>
>
> --
> Sean
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
Did more changes on zombie detector script and pushed them. Have now moved
master build over to use this zombie detector script only since it seems to
basically work (Translation: post-build, we no longer have script in a text
window up in jenkins for master build.... instead we call out to the
checked-in ./dev-support/zombie-detector.sh passing current build number).
Now I'm working on getting test-patch.sh to use the same zombie detector. I
just changed test-patch.sh to run zombie-detector before it does its old
stuff. Hopefully it keeps working.
At end of the day, will have all builds using same checked-in script rather
than running script out of jenkins configuration.
St.Ack
On Sat, Nov 7, 2015 at 11:41 AM, Stack <st...@duboce.net> wrote:
> Is this an example Sean,
> https://builds.apache.org/job/HBase-Trunk_matrix/442/jdk=latest1.7,label=Hadoop/consoleText
> ?
> Thanks,
> St.Ack
>
> On Thu, Nov 5, 2015 at 8:42 AM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> If Maven has trouble grabbing a pom but not an artifact, it'll
>> substitute in a placeholder pom that doesn't have e.g. license
>> information. That can result in a local repo that fails this way until
>> the refresh timeout hits for grabbing a pom again.
>>
>> On Wed, Nov 4, 2015 at 5:33 PM, Stack <st...@duboce.net> wrote:
>> > Thanks Andrew. Weird is that it is sporadic. Will keep an eye on it.
>> > St.Ack
>> >
>> > On Wed, Nov 4, 2015 at 8:14 AM, Andrew Purtell <
>> andrew.purtell@gmail.com>
>> > wrote:
>> >
>> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
>> >> java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
>> >>
>> >> This means a Velocity macro for building LICENSE info about a component
>> >> has failed because necessary information is missing in the Maven model.
>> >> When I have seen this it has been due to a change in dependencies
>> without
>> >> necessary updates to supplemental-models.xml. Running the LICENSE and
>> >> NOTICE aggregations in debug mode will print out clues as to what
>> >> specifically is missing. See defines in the POMs for doing so. (I'm
>> not at
>> >> the computer yet so can't pull up the specifics.)
>> >>
>> >>
>> >> > On Nov 4, 2015, at 6:23 AM, Stack <st...@duboce.net> wrote:
>> >> >
>> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
>> >> > java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
>> >>
>>
>>
>>
>> --
>> Sean
>>
>
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
Is this an example Sean,
https://builds.apache.org/job/HBase-Trunk_matrix/442/jdk=latest1.7,label=Hadoop/consoleText
?
Thanks,
St.Ack
On Thu, Nov 5, 2015 at 8:42 AM, Sean Busbey <bu...@cloudera.com> wrote:
> If Maven has trouble grabbing a pom but not an artifact, it'll
> substitute in a placeholder pom that doesn't have e.g. license
> information. That can result in a local repo that fails this way until
> the refresh timeout hits for grabbing a pom again.
>
> On Wed, Nov 4, 2015 at 5:33 PM, Stack <st...@duboce.net> wrote:
> > Thanks Andrew. Weird is that it is sporadic. Will keep an eye on it.
> > St.Ack
> >
> > On Wed, Nov 4, 2015 at 8:14 AM, Andrew Purtell <andrew.purtell@gmail.com
> >
> > wrote:
> >
> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
> >> java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
> >>
> >> This means a Velocity macro for building LICENSE info about a component
> >> has failed because necessary information is missing in the Maven model.
> >> When I have seen this it has been due to a change in dependencies
> without
> >> necessary updates to supplemental-models.xml. Running the LICENSE and
> >> NOTICE aggregations in debug mode will print out clues as to what
> >> specifically is missing. See defines in the POMs for doing so. (I'm not
> at
> >> the computer yet so can't pull up the specifics.)
> >>
> >>
> >> > On Nov 4, 2015, at 6:23 AM, Stack <st...@duboce.net> wrote:
> >> >
> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
> >> > java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
> >>
>
>
>
> --
> Sean
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
Thanks Sean. That helps.
FYI @here, am messing on trunk builds w/ post-build script section. I'll
probably mess it up a few times.... Be warned.
St.Ack
On Thu, Nov 5, 2015 at 8:42 AM, Sean Busbey <bu...@cloudera.com> wrote:
> If Maven has trouble grabbing a pom but not an artifact, it'll
> substitute in a placeholder pom that doesn't have e.g. license
> information. That can result in a local repo that fails this way until
> the refresh timeout hits for grabbing a pom again.
>
> On Wed, Nov 4, 2015 at 5:33 PM, Stack <st...@duboce.net> wrote:
> > Thanks Andrew. Weird is that it is sporadic. Will keep an eye on it.
> > St.Ack
> >
> > On Wed, Nov 4, 2015 at 8:14 AM, Andrew Purtell <andrew.purtell@gmail.com
> >
> > wrote:
> >
> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
> >> java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
> >>
> >> This means a Velocity macro for building LICENSE info about a component
> >> has failed because necessary information is missing in the Maven model.
> >> When I have seen this it has been due to a change in dependencies
> without
> >> necessary updates to supplemental-models.xml. Running the LICENSE and
> >> NOTICE aggregations in debug mode will print out clues as to what
> >> specifically is missing. See defines in the POMs for doing so. (I'm not
> at
> >> the computer yet so can't pull up the specifics.)
> >>
> >>
> >> > On Nov 4, 2015, at 6:23 AM, Stack <st...@duboce.net> wrote:
> >> >
> >> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
> >> > java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
> >>
>
>
>
> --
> Sean
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
If Maven has trouble grabbing a pom but not an artifact, it'll
substitute in a placeholder pom that doesn't have e.g. license
information. That can result in a local repo that fails this way until
the refresh timeout hits for grabbing a pom again.
On Wed, Nov 4, 2015 at 5:33 PM, Stack <st...@duboce.net> wrote:
> Thanks Andrew. Weird is that it is sporadic. Will keep an eye on it.
> St.Ack
>
> On Wed, Nov 4, 2015 at 8:14 AM, Andrew Purtell <an...@gmail.com>
> wrote:
>
>> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
>> java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
>>
>> This means a Velocity macro for building LICENSE info about a component
>> has failed because necessary information is missing in the Maven model.
>> When I have seen this it has been due to a change in dependencies without
>> necessary updates to supplemental-models.xml. Running the LICENSE and
>> NOTICE aggregations in debug mode will print out clues as to what
>> specifically is missing. See defines in the POMs for doing so. (I'm not at
>> the computer yet so can't pull up the specifics.)
>>
>>
>> > On Nov 4, 2015, at 6:23 AM, Stack <st...@duboce.net> wrote:
>> >
>> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
>> > java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
>>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
Thanks Andrew. Weird is that it is sporadic. Will keep an eye on it.
St.Ack
On Wed, Nov 4, 2015 at 8:14 AM, Andrew Purtell <an...@gmail.com>
wrote:
> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
> java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
>
> This means a Velocity macro for building LICENSE info about a component
> has failed because necessary information is missing in the Maven model.
> When I have seen this it has been due to a change in dependencies without
> necessary updates to supplemental-models.xml. Running the LICENSE and
> NOTICE aggregations in debug mode will print out clues as to what
> specifically is missing. See defines in the POMs for doing so. (I'm not at
> the computer yet so can't pull up the specifics.)
>
>
> > On Nov 4, 2015, at 6:23 AM, Stack <st...@duboce.net> wrote:
> >
> > [ERROR] Error invoking method 'get(java.lang.Integer)' in
> > java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Andrew Purtell <an...@gmail.com>.
> [ERROR] Error invoking method 'get(java.lang.Integer)' in java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
This means a Velocity macro for building LICENSE info about a component has failed because necessary information is missing in the Maven model. When I have seen this it has been due to a change in dependencies without necessary updates to supplemental-models.xml. Running the LICENSE and NOTICE aggregations in debug mode will print out clues as to what specifically is missing. See defines in the POMs for doing so. (I'm not at the computer yet so can't pull up the specifics.)
> On Nov 4, 2015, at 6:23 AM, Stack <st...@duboce.net> wrote:
>
> [ERROR] Error invoking method 'get(java.lang.Integer)' in
> java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
Clues on how to figure root cause of:
[ERROR] Error invoking method 'get(java.lang.Integer)' in
java.util.ArrayList at META-INF/LICENSE.vm[line 1627, column 22]
.... comes of the initial findbugs run when it goes into assembly. See here
in Trunk matrix:
https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/jdk=latest1.7,label=Hadoop/430/console
Then the checkstyle run fails with this in another 1.7 trunk matrix run:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-checkstyle-plugin:2.16:checkstyle
(default-cli) on project hbase-external-blockcache: An error has
occurred in Checkstyle report generation. Failed during checkstyle
execution: Unable to process suppressions file location:
hbase/checkstyle-suppressions.xml: Cannot create file-based
resource:invalid block type -> [Help 1]
Thanks,
St.Ack
On Tue, Nov 3, 2015 at 10:31 AM, Stack <st...@duboce.net> wrote:
> Maybe it got reenabled. I've disabled it now. Thanks Sean.
> St.Ack
>
> On Tue, Nov 3, 2015 at 9:50 AM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> I thought HBase_TRUNK was already disabled. if I missed it, disabling
>> should be sufficient. There are a few older non-matrix (0.98 I think?)
>> that we can delete once we're confident in the matrix version
>>
>> On Tue, Nov 3, 2015 at 8:44 AM, Stack <st...@duboce.net> wrote:
>> > Stuff seems to be basically working. I just committed a fix for this
>> > message so it should be gone now:
>> >
>> > grep: unknown devices method
>> > grep: write error: Broken pipe
>> >
>> >
>> > Hopefully we've seen last of the errant surefire kills.
>> >
>> > We have an HBASE_TRUNK build and we have an HBASE_TRUNK_MATRIX. We
>> should
>> > get rid of the former now?
>> >
>> > St.Ack
>> >
>> > On Mon, Nov 2, 2015 at 11:03 PM, Stack <st...@duboce.net> wrote:
>> >
>> >> I just did an edit of all of our jenkins configuration. I changed the
>> >> post-build zombie section. The surefire-killer was hanging out here it
>> >> seems (For detail, see HBASE-14589 Looking for the surefire-killer;
>> builds
>> >> being killed...). I may have messed up builds. Will fix in morning if
>> so
>> >> (currently builds seem a bit backed up so won't wait around).
>> >>
>> >> St.Ack
>> >>
>> >> On Fri, Oct 30, 2015 at 9:18 AM, Sean Busbey <bu...@cloudera.com>
>> wrote:
>> >>
>> >>> Hi Folks,
>> >>>
>> >>> Brief summary of cleanup/changes in our builds. ref thread '[DISCUSS]
>> >>> reducing our load on builds.a.o' on reasoning or if you have concerns.
>> >>>
>> >>> Our project job view now has a brief summary of what's going on:
>> >>>
>> >>> https://builds.apache.org/view/H-L/view/HBase/
>> >>>
>> >>> * Where we had matrix builds, I confirmed they're doing the same thing
>> >>> (in at least one test-configuration) as the non-matrix version
>> >>> * I disabled jobs that were duplicating work done in matrix jobs, or
>> >>> that only worked on a tag
>> >>> * Made sure all non-disabled 0.98+ jobs are set to notify jira +
>> >>> builds mailing list
>> >>> * I updated pre-1.1 jobs to look for changes 1/day
>> >>> * I updated 1.1 jobs to look for changes every 8 hours
>> >>> * I updated 1.2, 1.3, and trunk to look for changes every 4 hours
>> >>>
>> >>> There's still some non-pressing clean up work to do, so I'll file a
>> >>> jira later today just to make sure it's all recorded somewhere.
>> >>>
>> >>> On Mon, Jun 15, 2015 at 11:55 PM, Sean Busbey <bu...@cloudera.com>
>> >>> wrote:
>> >>> > Added new HBase-1.3 build or branch-1 and changed the existing 1.2
>> >>> builds to
>> >>> > use branch-1.2. (ref HBASE-13912)
>> >>> >
>> >>> > On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com>
>> >>> wrote:
>> >>> >>
>> >>> >> I've add a new build test to run the ITs under sunny day conditions
>> >>> using
>> >>> >> minicluster for both java 7 and java 8 on the 1.2 line.
>> >>> >>
>> >>> >> https://builds.apache.org/job/HBase-1.2-IT/
>> >>> >>
>> >>> >> I haven't turned on notifications yet, because BulkIngest and
>> >>> TestIngest
>> >>> >> are read. details on HBASE-13750
>> >>> >>
>> >>> >> On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com>
>> >>> wrote:
>> >>> >>>
>> >>> >>> sigh. that should have been "is now a matrix build".
>> >>> >>>
>> >>> >>> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <busbey@cloudera.com
>> >
>> >>> wrote:
>> >>> >>>>
>> >>> >>>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and
>> JDK8
>> >>> in
>> >>> >>>> parallel.
>> >>> >>>>
>> >>> >>>> I saved the old job for now and left it disabled[2].
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> [1]:
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
>> >>> >>>> [2]:
>> >>> >>>>
>> >>>
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>> >>> >>>>
>> >>> >>>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <
>> busbey@cloudera.com>
>> >>> >>>> wrote:
>> >>> >>>>>
>> >>> >>>>> Sorry for the noise. I also updated the checkstyle step on
>> >>> HBase-TRUNK
>> >>> >>>>>
>> >>> >>>>> from
>> >>> >>>>> mvn checkstyle:checkstyle
>> >>> >>>>> to
>> >>> >>>>> mvn -DskipTests package checkstyle:checkstyle
>> >>> >>>>>
>> >>> >>>>> to deal with the same issue. Looks all clear now.
>> >>> >>>>>
>> >>> >>>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <
>> busbey@cloudera.com>
>> >>> >>>>> wrote:
>> >>> >>>>>>
>> >>> >>>>>> Updated the following builds:
>> >>> >>>>>>
>> >>> >>>>>> * HBase-TRUNK
>> >>> >>>>>>
>> >>> >>>>>> moved from
>> >>> >>>>>>
>> >>> >>>>>> mvn clean compile findbugs:findbugs
>> >>> >>>>>>
>> >>> >>>>>> to
>> >>> >>>>>>
>> >>> >>>>>> mvn clean -DskipTests package findbugs:findbugs
>> >>> >>>>>>
>> >>> >>>>>> To work around the known issue where we can't do a bootstrap
>> >>> compile
>> >>> >>>>>> without previous install or remote SNAPSHOT artifacts.
>> (recently
>> >>> triggered
>> >>> >>>>>> by the addition of the procedure module on master)
>> >>> >>>>>>
>> >>> >>>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com>
>> wrote:
>> >>> >>>>>>>
>> >>> >>>>>>> Make findbugs and checkstyle plugins always run.
>> >>> >>>>>>> The default behavior only publishes result for stable builds,
>> but
>> >>> >>>>>>> master is
>> >>> >>>>>>> not always stable and sometimes we introduce new warnings in
>> >>> unstable
>> >>> >>>>>>> builds(see builds 6310-6314).
>> >>> >>>>>>> And findbugs and checkstyle will not fail unless we abort the
>> >>> >>>>>>> building
>> >>> >>>>>>> process I think, so it is safe to turn it on every time.
>> >>> >>>>>>>
>> >>> >>>>>>> Thanks.
>> >>> >>>>>>>
>> >>> >>>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>> >>> >>>>>>>
>> >>> >>>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>> >>> >>>>>>> >
>> >>> >>>>>>> > Cheers
>> >>> >>>>>>> >
>> >>> >>>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com>
>> >>> wrote:
>> >>> >>>>>>> >
>> >>> >>>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
>> >>> >>>>>>> > > findbugs,
>> >>> >>>>>>> > > checkstyle and jacoco coverage report.
>> >>> >>>>>>> > > The config has been tested on
>> >>> >>>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for
>> nearly
>> >>> 30
>> >>> >>>>>>> > > times.
>> >>> >>>>>>> > Not
>> >>> >>>>>>> > > much different from HBase-TRUNK unless it runs ~30%
>> slower(the
>> >>> >>>>>>> > > overhead
>> >>> >>>>>>> > of
>> >>> >>>>>>> > > collecting information for code coverage).
>> >>> >>>>>>> > > Thanks.
>> >>> >>>>>>> > >
>> >>> >>>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <
>> apurtell@apache.org
>> >>> >:
>> >>> >>>>>>> > >
>> >>> >>>>>>> > > > +1, thanks a lot for improving our build hygiene.
>> >>> >>>>>>> > > >
>> >>> >>>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar
>> >>> >>>>>>> > > > <en...@gmail.com>
>> >>> >>>>>>> > > wrote:
>> >>> >>>>>>> > > >
>> >>> >>>>>>> > > > > Thanks Sean. This is great.
>> >>> >>>>>>> > > > >
>> >>> >>>>>>> > > > > Enis
>> >>> >>>>>>> > > > >
>> >>> >>>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey
>> >>> >>>>>>> > > > > <bu...@cloudera.com>
>> >>> >>>>>>> > > > wrote:
>> >>> >>>>>>> > > > >
>> >>> >>>>>>> > > > > > FYI I just finished chasing down the breakage for
>> mvn
>> >>> site
>> >>> >>>>>>> > > > > > on all
>> >>> >>>>>>> > > patch
>> >>> >>>>>>> > > > > > builds.
>> >>> >>>>>>> > > > > >
>> >>> >>>>>>> > > > > > HBASE-13191 consolidates the few places in the
>> >>> test-patch
>> >>> >>>>>>> > > > > > code
>> >>> >>>>>>> > where
>> >>> >>>>>>> > > we
>> >>> >>>>>>> > > > > > hard coded MAVEN_OPTS.
>> >>> >>>>>>> > > > > >
>> >>> >>>>>>> > > > > > If you look at the PreCommit job now, we use the
>> "set
>> >>> >>>>>>> > > > > > environment
>> >>> >>>>>>> > > > > > variables" option to set MAVEN_OPTS and then
>> everything
>> >>> >>>>>>> > > > > > else
>> >>> >>>>>>> > respects
>> >>> >>>>>>> > > > > that
>> >>> >>>>>>> > > > > > setting.
>> >>> >>>>>>> > > > > >
>> >>> >>>>>>> > > > > > I set the initial value to be a combination of the
>> >>> memory
>> >>> >>>>>>> > limitations
>> >>> >>>>>>> > > > > we've
>> >>> >>>>>>> > > > > > been actually running with (the ~6G was getting
>> ignored)
>> >>> >>>>>>> > > > > > and the
>> >>> >>>>>>> > > > permgen
>> >>> >>>>>>> > > > > > needed for site.
>> >>> >>>>>>> > > > > >
>> >>> >>>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData
>> >>> -XX:MaxPermSize=256m
>> >>> >>>>>>> > > > > >
>> >>> >>>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <
>> >>> stack@duboce.net>
>> >>> >>>>>>> > > > > > wrote:
>> >>> >>>>>>> > > > > >
>> >>> >>>>>>> > > > > > > I just made TRUNK and branch-1 builds use same
>> jvm as
>> >>> >>>>>>> > > > > > > patch-build
>> >>> >>>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the
>> MAVEN_OPTS
>> >>> to
>> >>> >>>>>>> > > > > > > be the
>> >>> >>>>>>> > > same
>> >>> >>>>>>> > > > as
>> >>> >>>>>>> > > > > > > those of trunk build too, setting
>> >>> >>>>>>> > > > > > > MAVEN_OPTS="-Xmx6100m"... it
>> >>> >>>>>>> > had
>> >>> >>>>>>> > > > been
>> >>> >>>>>>> > > > > > > 3000.
>> >>> >>>>>>> > > > > > >
>> >>> >>>>>>> > > > > > > Yours,
>> >>> >>>>>>> > > > > > > St.Ack
>> >>> >>>>>>> > > > > > >
>> >>> >>>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <
>> >>> stack@duboce.net>
>> >>> >>>>>>> > > > > > > wrote:
>> >>> >>>>>>> > > > > > >
>> >>> >>>>>>> > > > > > > > I upped hadoopqa retention to keep last 100
>> builds
>> >>> and
>> >>> >>>>>>> > > > > > > > or last
>> >>> >>>>>>> > 7
>> >>> >>>>>>> > > > > days,
>> >>> >>>>>>> > > > > > > > whichever comes first.
>> >>> >>>>>>> > > > > > > > St.Ack
>> >>> >>>>>>> > > > > > > >
>> >>> >>>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack
>> >>> >>>>>>> > > > > > > > <st...@duboce.net>
>> >>> >>>>>>> > wrote:
>> >>> >>>>>>> > > > > > > >
>> >>> >>>>>>> > > > > > > >> Branch-1 and master have stabilized and now run
>> >>> mostly
>> >>> >>>>>>> > > > > > > >> blue
>> >>> >>>>>>> > > (give
>> >>> >>>>>>> > > > or
>> >>> >>>>>>> > > > > > > take
>> >>> >>>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue
>> >>> branch-1
>> >>> >>>>>>> > > > > > > >> has
>> >>> >>>>>>> > > helped
>> >>> >>>>>>> > > > us
>> >>> >>>>>>> > > > > > > >> identify at least one destabilizing commit in
>> the
>> >>> last
>> >>> >>>>>>> > > > > > > >> few
>> >>> >>>>>>> > days,
>> >>> >>>>>>> > > > > maybe
>> >>> >>>>>>> > > > > > > two;
>> >>> >>>>>>> > > > > > > >> this is as it should be (smile).
>> >>> >>>>>>> > > > > > > >>
>> >>> >>>>>>> > > > > > > >> Lets keep our builds blue. If you commit a
>> patch,
>> >>> make
>> >>> >>>>>>> > > > > > > >> sure
>> >>> >>>>>>> > > > > subsequent
>> >>> >>>>>>> > > > > > > >> builds stay blue. You can subscribe to
>> >>> >>>>>>> > builds@hbase.apache.org
>> >>> >>>>>>> > > to
>> >>> >>>>>>> > > > > get
>> >>> >>>>>>> > > > > > > >> notice of failures if not already subscribed.
>> >>> >>>>>>> > > > > > > >>
>> >>> >>>>>>> > > > > > > >> Thanks,
>> >>> >>>>>>> > > > > > > >> St.Ack
>> >>> >>>>>>> > > > > > > >>
>> >>> >>>>>>> > > > > > > >> 1.
>> >>> >>>>>>> >
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>> >>> >>>>>>> > > > > > > >> 2.
>> >>> >>>>>>> > >
>> >>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>> >>> >>>>>>> > > > > > > >>
>> >>> >>>>>>> > > > > > > >>
>> >>> >>>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack
>> >>> >>>>>>> > > > > > > >> <st...@duboce.net>
>> >>> >>>>>>> > > wrote:
>> >>> >>>>>>> > > > > > > >>
>> >>> >>>>>>> > > > > > > >>> A few notes on testing.
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>> Too long to read, infra is more capable now
>> and
>> >>> after
>> >>> >>>>>>> > > > > > > >>> some
>> >>> >>>>>>> > > work,
>> >>> >>>>>>> > > > we
>> >>> >>>>>>> > > > > > are
>> >>> >>>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue.
>> >>> Lets
>> >>> >>>>>>> > > > > > > >>> try and
>> >>> >>>>>>> > > keep
>> >>> >>>>>>> > > > it
>> >>> >>>>>>> > > > > > > this
>> >>> >>>>>>> > > > > > > >>> way going forward.
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>> A recent spurt of test fixing combined with
>> more
>> >>> >>>>>>> > > > > > > >>> capable
>> >>> >>>>>>> > > hardware
>> >>> >>>>>>> > > > > > seems
>> >>> >>>>>>> > > > > > > >>> to have gotten us to a new place; tests are
>> mostly
>> >>> >>>>>>> > > > > > > >>> passing
>> >>> >>>>>>> > now
>> >>> >>>>>>> > > on
>> >>> >>>>>>> > > > > > > branch-1
>> >>> >>>>>>> > > > > > > >>> and master. Lets try and keep it this way and
>> >>> start
>> >>> >>>>>>> > > > > > > >>> to trust
>> >>> >>>>>>> > > our
>> >>> >>>>>>> > > > > > test
>> >>> >>>>>>> > > > > > > runs
>> >>> >>>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try
>> and
>> >>> nail
>> >>> >>>>>>> > > > > > > >>> them.
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>> Our tests now run in parallel with other test
>> >>> suites
>> >>> >>>>>>> > > > > > > >>> where
>> >>> >>>>>>> > > > previous
>> >>> >>>>>>> > > > > > we
>> >>> >>>>>>> > > > > > > >>> ran alone. You can see this sometimes when our
>> >>> zombie
>> >>> >>>>>>> > detector
>> >>> >>>>>>> > > > > > reports
>> >>> >>>>>>> > > > > > > >>> tests from another project altogether as
>> lingerers
>> >>> >>>>>>> > > > > > > >>> (To be
>> >>> >>>>>>> > > fixed).
>> >>> >>>>>>> > > > > > > Some of
>> >>> >>>>>>> > > > > > > >>> our tests are failing because a concurrent
>> hbase
>> >>> run
>> >>> >>>>>>> > > > > > > >>> is
>> >>> >>>>>>> > undoing
>> >>> >>>>>>> > > > > > > classes and
>> >>> >>>>>>> > > > > > > >>> data from under it. Also, lets fix.
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for
>> >>> them to
>> >>> >>>>>>> > complete.
>> >>> >>>>>>> > > > > Many
>> >>> >>>>>>> > > > > > > >>> are heavy-duty integration tests starting up
>> >>> multiple
>> >>> >>>>>>> > clusters
>> >>> >>>>>>> > > > and
>> >>> >>>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle
>> they
>> >>> >>>>>>> > > > > > > >>> pass at
>> >>> >>>>>>> > all.
>> >>> >>>>>>> > > > > > > Usually
>> >>> >>>>>>> > > > > > > >>> integration tests have been cast as unit tests
>> >>> >>>>>>> > > > > > > >>> because there
>> >>> >>>>>>> > > was
>> >>> >>>>>>> > > > no
>> >>> >>>>>>> > > > > > > where
>> >>> >>>>>>> > > > > > > >>> else for them to get an airing. We have the
>> >>> hbase-it
>> >>> >>>>>>> > > > > > > >>> suite
>> >>> >>>>>>> > now
>> >>> >>>>>>> > > > > which
>> >>> >>>>>>> > > > > > > would
>> >>> >>>>>>> > > > > > > >>> be a more apt place but until these are run
>> on a
>> >>> >>>>>>> > > > > > > >>> regular
>> >>> >>>>>>> > basis
>> >>> >>>>>>> > > in
>> >>> >>>>>>> > > > > > > public
>> >>> >>>>>>> > > > > > > >>> for all to see, the fat integration tests
>> >>> disguised
>> >>> >>>>>>> > > > > > > >>> as unit
>> >>> >>>>>>> > > tests
>> >>> >>>>>>> > > > > > will
>> >>> >>>>>>> > > > > > > >>> remain. A review of our current unit tests
>> >>> weeding
>> >>> >>>>>>> > > > > > > >>> the old
>> >>> >>>>>>> > > cruft
>> >>> >>>>>>> > > > > and
>> >>> >>>>>>> > > > > > > the
>> >>> >>>>>>> > > > > > > >>> no longer relevant or duplicates would be a
>> nice
>> >>> >>>>>>> > > > > > > >>> undertaking
>> >>> >>>>>>> > if
>> >>> >>>>>>> > > > > > > someone is
>> >>> >>>>>>> > > > > > > >>> looking to contribute.
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>> Alex Newman has been working on making our
>> tests
>> >>> work
>> >>> >>>>>>> > > > > > > >>> up on
>> >>> >>>>>>> > > > travis
>> >>> >>>>>>> > > > > > and
>> >>> >>>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes
>> >>> end-to-end.
>> >>> >>>>>>> > > > > > > >>> He
>> >>> >>>>>>> > also
>> >>> >>>>>>> > > > > added
>> >>> >>>>>>> > > > > > in
>> >>> >>>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
>> >>> >>>>>>> > > > > > > >>> mapreduce --
>> >>> >>>>>>> > > > > alongside
>> >>> >>>>>>> > > > > > > our
>> >>> >>>>>>> > > > > > > >>> old "sizing" categorizations of
>> >>> small/medium/large.
>> >>> >>>>>>> > > > > > > >>> His
>> >>> >>>>>>> > > thinking
>> >>> >>>>>>> > > > > is
>> >>> >>>>>>> > > > > > > that
>> >>> >>>>>>> > > > > > > >>> we can run these categorizations in parallel
>> so we
>> >>> >>>>>>> > > > > > > >>> could run
>> >>> >>>>>>> > > the
>> >>> >>>>>>> > > > > > total
>> >>> >>>>>>> > > > > > > >>> suite in about the time of the longest test,
>> say
>> >>> >>>>>>> > 20-30minutes?
>> >>> >>>>>>> > > > We
>> >>> >>>>>>> > > > > > > could
>> >>> >>>>>>> > > > > > > >>> even change Apache to run them this way.
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>> FYI,
>> >>> >>>>>>> > > > > > > >>> St.Ack
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>>
>> >>> >>>>>>> > > > > > > >>
>> >>> >>>>>>> > > > > > > >
>> >>> >>>>>>> > > > > > >
>> >>> >>>>>>> > > > > >
>> >>> >>>>>>> > > > > >
>> >>> >>>>>>> > > > > >
>> >>> >>>>>>> > > > > > --
>> >>> >>>>>>> > > > > > Sean
>> >>> >>>>>>> > > > > >
>> >>> >>>>>>> > > > >
>> >>> >>>>>>> > > >
>> >>> >>>>>>> > > >
>> >>> >>>>>>> > > >
>> >>> >>>>>>> > > > --
>> >>> >>>>>>> > > > Best regards,
>> >>> >>>>>>> > > >
>> >>> >>>>>>> > > > - Andy
>> >>> >>>>>>> > > >
>> >>> >>>>>>> > > > Problems worthy of attack prove their worth by hitting
>> >>> back. -
>> >>> >>>>>>> > > > Piet
>> >>> >>>>>>> > Hein
>> >>> >>>>>>> > > > (via Tom White)
>> >>> >>>>>>> > > >
>> >>> >>>>>>> > >
>> >>> >>>>>>> >
>> >>> >>>>>>
>> >>> >>>>>>
>> >>> >>>>>>
>> >>> >>>>>>
>> >>> >>>>>> --
>> >>> >>>>>> Sean
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> --
>> >>> >>>>> Sean
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> --
>> >>> >>>> Sean
>> >>> >>>
>> >>> >>>
>> >>> >>>
>> >>> >>>
>> >>> >>> --
>> >>> >>> Sean
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Sean
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Sean
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Sean
>> >>>
>> >>
>> >>
>>
>>
>>
>> --
>> Sean
>>
>
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
Maybe it got reenabled. I've disabled it now. Thanks Sean.
St.Ack
On Tue, Nov 3, 2015 at 9:50 AM, Sean Busbey <bu...@cloudera.com> wrote:
> I thought HBase_TRUNK was already disabled. if I missed it, disabling
> should be sufficient. There are a few older non-matrix (0.98 I think?)
> that we can delete once we're confident in the matrix version
>
> On Tue, Nov 3, 2015 at 8:44 AM, Stack <st...@duboce.net> wrote:
> > Stuff seems to be basically working. I just committed a fix for this
> > message so it should be gone now:
> >
> > grep: unknown devices method
> > grep: write error: Broken pipe
> >
> >
> > Hopefully we've seen last of the errant surefire kills.
> >
> > We have an HBASE_TRUNK build and we have an HBASE_TRUNK_MATRIX. We should
> > get rid of the former now?
> >
> > St.Ack
> >
> > On Mon, Nov 2, 2015 at 11:03 PM, Stack <st...@duboce.net> wrote:
> >
> >> I just did an edit of all of our jenkins configuration. I changed the
> >> post-build zombie section. The surefire-killer was hanging out here it
> >> seems (For detail, see HBASE-14589 Looking for the surefire-killer;
> builds
> >> being killed...). I may have messed up builds. Will fix in morning if so
> >> (currently builds seem a bit backed up so won't wait around).
> >>
> >> St.Ack
> >>
> >> On Fri, Oct 30, 2015 at 9:18 AM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >>
> >>> Hi Folks,
> >>>
> >>> Brief summary of cleanup/changes in our builds. ref thread '[DISCUSS]
> >>> reducing our load on builds.a.o' on reasoning or if you have concerns.
> >>>
> >>> Our project job view now has a brief summary of what's going on:
> >>>
> >>> https://builds.apache.org/view/H-L/view/HBase/
> >>>
> >>> * Where we had matrix builds, I confirmed they're doing the same thing
> >>> (in at least one test-configuration) as the non-matrix version
> >>> * I disabled jobs that were duplicating work done in matrix jobs, or
> >>> that only worked on a tag
> >>> * Made sure all non-disabled 0.98+ jobs are set to notify jira +
> >>> builds mailing list
> >>> * I updated pre-1.1 jobs to look for changes 1/day
> >>> * I updated 1.1 jobs to look for changes every 8 hours
> >>> * I updated 1.2, 1.3, and trunk to look for changes every 4 hours
> >>>
> >>> There's still some non-pressing clean up work to do, so I'll file a
> >>> jira later today just to make sure it's all recorded somewhere.
> >>>
> >>> On Mon, Jun 15, 2015 at 11:55 PM, Sean Busbey <bu...@cloudera.com>
> >>> wrote:
> >>> > Added new HBase-1.3 build or branch-1 and changed the existing 1.2
> >>> builds to
> >>> > use branch-1.2. (ref HBASE-13912)
> >>> >
> >>> > On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com>
> >>> wrote:
> >>> >>
> >>> >> I've add a new build test to run the ITs under sunny day conditions
> >>> using
> >>> >> minicluster for both java 7 and java 8 on the 1.2 line.
> >>> >>
> >>> >> https://builds.apache.org/job/HBase-1.2-IT/
> >>> >>
> >>> >> I haven't turned on notifications yet, because BulkIngest and
> >>> TestIngest
> >>> >> are read. details on HBASE-13750
> >>> >>
> >>> >> On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com>
> >>> wrote:
> >>> >>>
> >>> >>> sigh. that should have been "is now a matrix build".
> >>> >>>
> >>> >>> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com>
> >>> wrote:
> >>> >>>>
> >>> >>>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and
> JDK8
> >>> in
> >>> >>>> parallel.
> >>> >>>>
> >>> >>>> I saved the old job for now and left it disabled[2].
> >>> >>>>
> >>> >>>>
> >>> >>>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
> >>> >>>> [2]:
> >>> >>>>
> >>>
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
> >>> >>>>
> >>> >>>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <busbey@cloudera.com
> >
> >>> >>>> wrote:
> >>> >>>>>
> >>> >>>>> Sorry for the noise. I also updated the checkstyle step on
> >>> HBase-TRUNK
> >>> >>>>>
> >>> >>>>> from
> >>> >>>>> mvn checkstyle:checkstyle
> >>> >>>>> to
> >>> >>>>> mvn -DskipTests package checkstyle:checkstyle
> >>> >>>>>
> >>> >>>>> to deal with the same issue. Looks all clear now.
> >>> >>>>>
> >>> >>>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <
> busbey@cloudera.com>
> >>> >>>>> wrote:
> >>> >>>>>>
> >>> >>>>>> Updated the following builds:
> >>> >>>>>>
> >>> >>>>>> * HBase-TRUNK
> >>> >>>>>>
> >>> >>>>>> moved from
> >>> >>>>>>
> >>> >>>>>> mvn clean compile findbugs:findbugs
> >>> >>>>>>
> >>> >>>>>> to
> >>> >>>>>>
> >>> >>>>>> mvn clean -DskipTests package findbugs:findbugs
> >>> >>>>>>
> >>> >>>>>> To work around the known issue where we can't do a bootstrap
> >>> compile
> >>> >>>>>> without previous install or remote SNAPSHOT artifacts. (recently
> >>> triggered
> >>> >>>>>> by the addition of the procedure module on master)
> >>> >>>>>>
> >>> >>>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com>
> wrote:
> >>> >>>>>>>
> >>> >>>>>>> Make findbugs and checkstyle plugins always run.
> >>> >>>>>>> The default behavior only publishes result for stable builds,
> but
> >>> >>>>>>> master is
> >>> >>>>>>> not always stable and sometimes we introduce new warnings in
> >>> unstable
> >>> >>>>>>> builds(see builds 6310-6314).
> >>> >>>>>>> And findbugs and checkstyle will not fail unless we abort the
> >>> >>>>>>> building
> >>> >>>>>>> process I think, so it is safe to turn it on every time.
> >>> >>>>>>>
> >>> >>>>>>> Thanks.
> >>> >>>>>>>
> >>> >>>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
> >>> >>>>>>>
> >>> >>>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
> >>> >>>>>>> >
> >>> >>>>>>> > Cheers
> >>> >>>>>>> >
> >>> >>>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com>
> >>> wrote:
> >>> >>>>>>> >
> >>> >>>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
> >>> >>>>>>> > > findbugs,
> >>> >>>>>>> > > checkstyle and jacoco coverage report.
> >>> >>>>>>> > > The config has been tested on
> >>> >>>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for
> nearly
> >>> 30
> >>> >>>>>>> > > times.
> >>> >>>>>>> > Not
> >>> >>>>>>> > > much different from HBase-TRUNK unless it runs ~30%
> slower(the
> >>> >>>>>>> > > overhead
> >>> >>>>>>> > of
> >>> >>>>>>> > > collecting information for code coverage).
> >>> >>>>>>> > > Thanks.
> >>> >>>>>>> > >
> >>> >>>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <
> apurtell@apache.org
> >>> >:
> >>> >>>>>>> > >
> >>> >>>>>>> > > > +1, thanks a lot for improving our build hygiene.
> >>> >>>>>>> > > >
> >>> >>>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar
> >>> >>>>>>> > > > <en...@gmail.com>
> >>> >>>>>>> > > wrote:
> >>> >>>>>>> > > >
> >>> >>>>>>> > > > > Thanks Sean. This is great.
> >>> >>>>>>> > > > >
> >>> >>>>>>> > > > > Enis
> >>> >>>>>>> > > > >
> >>> >>>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey
> >>> >>>>>>> > > > > <bu...@cloudera.com>
> >>> >>>>>>> > > > wrote:
> >>> >>>>>>> > > > >
> >>> >>>>>>> > > > > > FYI I just finished chasing down the breakage for mvn
> >>> site
> >>> >>>>>>> > > > > > on all
> >>> >>>>>>> > > patch
> >>> >>>>>>> > > > > > builds.
> >>> >>>>>>> > > > > >
> >>> >>>>>>> > > > > > HBASE-13191 consolidates the few places in the
> >>> test-patch
> >>> >>>>>>> > > > > > code
> >>> >>>>>>> > where
> >>> >>>>>>> > > we
> >>> >>>>>>> > > > > > hard coded MAVEN_OPTS.
> >>> >>>>>>> > > > > >
> >>> >>>>>>> > > > > > If you look at the PreCommit job now, we use the "set
> >>> >>>>>>> > > > > > environment
> >>> >>>>>>> > > > > > variables" option to set MAVEN_OPTS and then
> everything
> >>> >>>>>>> > > > > > else
> >>> >>>>>>> > respects
> >>> >>>>>>> > > > > that
> >>> >>>>>>> > > > > > setting.
> >>> >>>>>>> > > > > >
> >>> >>>>>>> > > > > > I set the initial value to be a combination of the
> >>> memory
> >>> >>>>>>> > limitations
> >>> >>>>>>> > > > > we've
> >>> >>>>>>> > > > > > been actually running with (the ~6G was getting
> ignored)
> >>> >>>>>>> > > > > > and the
> >>> >>>>>>> > > > permgen
> >>> >>>>>>> > > > > > needed for site.
> >>> >>>>>>> > > > > >
> >>> >>>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData
> >>> -XX:MaxPermSize=256m
> >>> >>>>>>> > > > > >
> >>> >>>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <
> >>> stack@duboce.net>
> >>> >>>>>>> > > > > > wrote:
> >>> >>>>>>> > > > > >
> >>> >>>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm
> as
> >>> >>>>>>> > > > > > > patch-build
> >>> >>>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the
> MAVEN_OPTS
> >>> to
> >>> >>>>>>> > > > > > > be the
> >>> >>>>>>> > > same
> >>> >>>>>>> > > > as
> >>> >>>>>>> > > > > > > those of trunk build too, setting
> >>> >>>>>>> > > > > > > MAVEN_OPTS="-Xmx6100m"... it
> >>> >>>>>>> > had
> >>> >>>>>>> > > > been
> >>> >>>>>>> > > > > > > 3000.
> >>> >>>>>>> > > > > > >
> >>> >>>>>>> > > > > > > Yours,
> >>> >>>>>>> > > > > > > St.Ack
> >>> >>>>>>> > > > > > >
> >>> >>>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <
> >>> stack@duboce.net>
> >>> >>>>>>> > > > > > > wrote:
> >>> >>>>>>> > > > > > >
> >>> >>>>>>> > > > > > > > I upped hadoopqa retention to keep last 100
> builds
> >>> and
> >>> >>>>>>> > > > > > > > or last
> >>> >>>>>>> > 7
> >>> >>>>>>> > > > > days,
> >>> >>>>>>> > > > > > > > whichever comes first.
> >>> >>>>>>> > > > > > > > St.Ack
> >>> >>>>>>> > > > > > > >
> >>> >>>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack
> >>> >>>>>>> > > > > > > > <st...@duboce.net>
> >>> >>>>>>> > wrote:
> >>> >>>>>>> > > > > > > >
> >>> >>>>>>> > > > > > > >> Branch-1 and master have stabilized and now run
> >>> mostly
> >>> >>>>>>> > > > > > > >> blue
> >>> >>>>>>> > > (give
> >>> >>>>>>> > > > or
> >>> >>>>>>> > > > > > > take
> >>> >>>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue
> >>> branch-1
> >>> >>>>>>> > > > > > > >> has
> >>> >>>>>>> > > helped
> >>> >>>>>>> > > > us
> >>> >>>>>>> > > > > > > >> identify at least one destabilizing commit in
> the
> >>> last
> >>> >>>>>>> > > > > > > >> few
> >>> >>>>>>> > days,
> >>> >>>>>>> > > > > maybe
> >>> >>>>>>> > > > > > > two;
> >>> >>>>>>> > > > > > > >> this is as it should be (smile).
> >>> >>>>>>> > > > > > > >>
> >>> >>>>>>> > > > > > > >> Lets keep our builds blue. If you commit a
> patch,
> >>> make
> >>> >>>>>>> > > > > > > >> sure
> >>> >>>>>>> > > > > subsequent
> >>> >>>>>>> > > > > > > >> builds stay blue. You can subscribe to
> >>> >>>>>>> > builds@hbase.apache.org
> >>> >>>>>>> > > to
> >>> >>>>>>> > > > > get
> >>> >>>>>>> > > > > > > >> notice of failures if not already subscribed.
> >>> >>>>>>> > > > > > > >>
> >>> >>>>>>> > > > > > > >> Thanks,
> >>> >>>>>>> > > > > > > >> St.Ack
> >>> >>>>>>> > > > > > > >>
> >>> >>>>>>> > > > > > > >> 1.
> >>> >>>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> >>> >>>>>>> > > > > > > >> 2.
> >>> >>>>>>> > >
> >>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> >>> >>>>>>> > > > > > > >>
> >>> >>>>>>> > > > > > > >>
> >>> >>>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack
> >>> >>>>>>> > > > > > > >> <st...@duboce.net>
> >>> >>>>>>> > > wrote:
> >>> >>>>>>> > > > > > > >>
> >>> >>>>>>> > > > > > > >>> A few notes on testing.
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>> Too long to read, infra is more capable now and
> >>> after
> >>> >>>>>>> > > > > > > >>> some
> >>> >>>>>>> > > work,
> >>> >>>>>>> > > > we
> >>> >>>>>>> > > > > > are
> >>> >>>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue.
> >>> Lets
> >>> >>>>>>> > > > > > > >>> try and
> >>> >>>>>>> > > keep
> >>> >>>>>>> > > > it
> >>> >>>>>>> > > > > > > this
> >>> >>>>>>> > > > > > > >>> way going forward.
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>> A recent spurt of test fixing combined with
> more
> >>> >>>>>>> > > > > > > >>> capable
> >>> >>>>>>> > > hardware
> >>> >>>>>>> > > > > > seems
> >>> >>>>>>> > > > > > > >>> to have gotten us to a new place; tests are
> mostly
> >>> >>>>>>> > > > > > > >>> passing
> >>> >>>>>>> > now
> >>> >>>>>>> > > on
> >>> >>>>>>> > > > > > > branch-1
> >>> >>>>>>> > > > > > > >>> and master. Lets try and keep it this way and
> >>> start
> >>> >>>>>>> > > > > > > >>> to trust
> >>> >>>>>>> > > our
> >>> >>>>>>> > > > > > test
> >>> >>>>>>> > > > > > > runs
> >>> >>>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try
> and
> >>> nail
> >>> >>>>>>> > > > > > > >>> them.
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>> Our tests now run in parallel with other test
> >>> suites
> >>> >>>>>>> > > > > > > >>> where
> >>> >>>>>>> > > > previous
> >>> >>>>>>> > > > > > we
> >>> >>>>>>> > > > > > > >>> ran alone. You can see this sometimes when our
> >>> zombie
> >>> >>>>>>> > detector
> >>> >>>>>>> > > > > > reports
> >>> >>>>>>> > > > > > > >>> tests from another project altogether as
> lingerers
> >>> >>>>>>> > > > > > > >>> (To be
> >>> >>>>>>> > > fixed).
> >>> >>>>>>> > > > > > > Some of
> >>> >>>>>>> > > > > > > >>> our tests are failing because a concurrent
> hbase
> >>> run
> >>> >>>>>>> > > > > > > >>> is
> >>> >>>>>>> > undoing
> >>> >>>>>>> > > > > > > classes and
> >>> >>>>>>> > > > > > > >>> data from under it. Also, lets fix.
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for
> >>> them to
> >>> >>>>>>> > complete.
> >>> >>>>>>> > > > > Many
> >>> >>>>>>> > > > > > > >>> are heavy-duty integration tests starting up
> >>> multiple
> >>> >>>>>>> > clusters
> >>> >>>>>>> > > > and
> >>> >>>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle
> they
> >>> >>>>>>> > > > > > > >>> pass at
> >>> >>>>>>> > all.
> >>> >>>>>>> > > > > > > Usually
> >>> >>>>>>> > > > > > > >>> integration tests have been cast as unit tests
> >>> >>>>>>> > > > > > > >>> because there
> >>> >>>>>>> > > was
> >>> >>>>>>> > > > no
> >>> >>>>>>> > > > > > > where
> >>> >>>>>>> > > > > > > >>> else for them to get an airing. We have the
> >>> hbase-it
> >>> >>>>>>> > > > > > > >>> suite
> >>> >>>>>>> > now
> >>> >>>>>>> > > > > which
> >>> >>>>>>> > > > > > > would
> >>> >>>>>>> > > > > > > >>> be a more apt place but until these are run on
> a
> >>> >>>>>>> > > > > > > >>> regular
> >>> >>>>>>> > basis
> >>> >>>>>>> > > in
> >>> >>>>>>> > > > > > > public
> >>> >>>>>>> > > > > > > >>> for all to see, the fat integration tests
> >>> disguised
> >>> >>>>>>> > > > > > > >>> as unit
> >>> >>>>>>> > > tests
> >>> >>>>>>> > > > > > will
> >>> >>>>>>> > > > > > > >>> remain. A review of our current unit tests
> >>> weeding
> >>> >>>>>>> > > > > > > >>> the old
> >>> >>>>>>> > > cruft
> >>> >>>>>>> > > > > and
> >>> >>>>>>> > > > > > > the
> >>> >>>>>>> > > > > > > >>> no longer relevant or duplicates would be a
> nice
> >>> >>>>>>> > > > > > > >>> undertaking
> >>> >>>>>>> > if
> >>> >>>>>>> > > > > > > someone is
> >>> >>>>>>> > > > > > > >>> looking to contribute.
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>> Alex Newman has been working on making our
> tests
> >>> work
> >>> >>>>>>> > > > > > > >>> up on
> >>> >>>>>>> > > > travis
> >>> >>>>>>> > > > > > and
> >>> >>>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes
> >>> end-to-end.
> >>> >>>>>>> > > > > > > >>> He
> >>> >>>>>>> > also
> >>> >>>>>>> > > > > added
> >>> >>>>>>> > > > > > in
> >>> >>>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
> >>> >>>>>>> > > > > > > >>> mapreduce --
> >>> >>>>>>> > > > > alongside
> >>> >>>>>>> > > > > > > our
> >>> >>>>>>> > > > > > > >>> old "sizing" categorizations of
> >>> small/medium/large.
> >>> >>>>>>> > > > > > > >>> His
> >>> >>>>>>> > > thinking
> >>> >>>>>>> > > > > is
> >>> >>>>>>> > > > > > > that
> >>> >>>>>>> > > > > > > >>> we can run these categorizations in parallel
> so we
> >>> >>>>>>> > > > > > > >>> could run
> >>> >>>>>>> > > the
> >>> >>>>>>> > > > > > total
> >>> >>>>>>> > > > > > > >>> suite in about the time of the longest test,
> say
> >>> >>>>>>> > 20-30minutes?
> >>> >>>>>>> > > > We
> >>> >>>>>>> > > > > > > could
> >>> >>>>>>> > > > > > > >>> even change Apache to run them this way.
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>> FYI,
> >>> >>>>>>> > > > > > > >>> St.Ack
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>>
> >>> >>>>>>> > > > > > > >>
> >>> >>>>>>> > > > > > > >
> >>> >>>>>>> > > > > > >
> >>> >>>>>>> > > > > >
> >>> >>>>>>> > > > > >
> >>> >>>>>>> > > > > >
> >>> >>>>>>> > > > > > --
> >>> >>>>>>> > > > > > Sean
> >>> >>>>>>> > > > > >
> >>> >>>>>>> > > > >
> >>> >>>>>>> > > >
> >>> >>>>>>> > > >
> >>> >>>>>>> > > >
> >>> >>>>>>> > > > --
> >>> >>>>>>> > > > Best regards,
> >>> >>>>>>> > > >
> >>> >>>>>>> > > > - Andy
> >>> >>>>>>> > > >
> >>> >>>>>>> > > > Problems worthy of attack prove their worth by hitting
> >>> back. -
> >>> >>>>>>> > > > Piet
> >>> >>>>>>> > Hein
> >>> >>>>>>> > > > (via Tom White)
> >>> >>>>>>> > > >
> >>> >>>>>>> > >
> >>> >>>>>>> >
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> --
> >>> >>>>>> Sean
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> --
> >>> >>>>> Sean
> >>> >>>>
> >>> >>>>
> >>> >>>>
> >>> >>>>
> >>> >>>> --
> >>> >>>> Sean
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> Sean
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Sean
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Sean
> >>>
> >>>
> >>>
> >>> --
> >>> Sean
> >>>
> >>
> >>
>
>
>
> --
> Sean
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
I thought HBase_TRUNK was already disabled. if I missed it, disabling
should be sufficient. There are a few older non-matrix (0.98 I think?)
that we can delete once we're confident in the matrix version
On Tue, Nov 3, 2015 at 8:44 AM, Stack <st...@duboce.net> wrote:
> Stuff seems to be basically working. I just committed a fix for this
> message so it should be gone now:
>
> grep: unknown devices method
> grep: write error: Broken pipe
>
>
> Hopefully we've seen last of the errant surefire kills.
>
> We have an HBASE_TRUNK build and we have an HBASE_TRUNK_MATRIX. We should
> get rid of the former now?
>
> St.Ack
>
> On Mon, Nov 2, 2015 at 11:03 PM, Stack <st...@duboce.net> wrote:
>
>> I just did an edit of all of our jenkins configuration. I changed the
>> post-build zombie section. The surefire-killer was hanging out here it
>> seems (For detail, see HBASE-14589 Looking for the surefire-killer; builds
>> being killed...). I may have messed up builds. Will fix in morning if so
>> (currently builds seem a bit backed up so won't wait around).
>>
>> St.Ack
>>
>> On Fri, Oct 30, 2015 at 9:18 AM, Sean Busbey <bu...@cloudera.com> wrote:
>>
>>> Hi Folks,
>>>
>>> Brief summary of cleanup/changes in our builds. ref thread '[DISCUSS]
>>> reducing our load on builds.a.o' on reasoning or if you have concerns.
>>>
>>> Our project job view now has a brief summary of what's going on:
>>>
>>> https://builds.apache.org/view/H-L/view/HBase/
>>>
>>> * Where we had matrix builds, I confirmed they're doing the same thing
>>> (in at least one test-configuration) as the non-matrix version
>>> * I disabled jobs that were duplicating work done in matrix jobs, or
>>> that only worked on a tag
>>> * Made sure all non-disabled 0.98+ jobs are set to notify jira +
>>> builds mailing list
>>> * I updated pre-1.1 jobs to look for changes 1/day
>>> * I updated 1.1 jobs to look for changes every 8 hours
>>> * I updated 1.2, 1.3, and trunk to look for changes every 4 hours
>>>
>>> There's still some non-pressing clean up work to do, so I'll file a
>>> jira later today just to make sure it's all recorded somewhere.
>>>
>>> On Mon, Jun 15, 2015 at 11:55 PM, Sean Busbey <bu...@cloudera.com>
>>> wrote:
>>> > Added new HBase-1.3 build or branch-1 and changed the existing 1.2
>>> builds to
>>> > use branch-1.2. (ref HBASE-13912)
>>> >
>>> > On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com>
>>> wrote:
>>> >>
>>> >> I've add a new build test to run the ITs under sunny day conditions
>>> using
>>> >> minicluster for both java 7 and java 8 on the 1.2 line.
>>> >>
>>> >> https://builds.apache.org/job/HBase-1.2-IT/
>>> >>
>>> >> I haven't turned on notifications yet, because BulkIngest and
>>> TestIngest
>>> >> are read. details on HBASE-13750
>>> >>
>>> >> On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com>
>>> wrote:
>>> >>>
>>> >>> sigh. that should have been "is now a matrix build".
>>> >>>
>>> >>> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com>
>>> wrote:
>>> >>>>
>>> >>>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8
>>> in
>>> >>>> parallel.
>>> >>>>
>>> >>>> I saved the old job for now and left it disabled[2].
>>> >>>>
>>> >>>>
>>> >>>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
>>> >>>> [2]:
>>> >>>>
>>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>>> >>>>
>>> >>>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com>
>>> >>>> wrote:
>>> >>>>>
>>> >>>>> Sorry for the noise. I also updated the checkstyle step on
>>> HBase-TRUNK
>>> >>>>>
>>> >>>>> from
>>> >>>>> mvn checkstyle:checkstyle
>>> >>>>> to
>>> >>>>> mvn -DskipTests package checkstyle:checkstyle
>>> >>>>>
>>> >>>>> to deal with the same issue. Looks all clear now.
>>> >>>>>
>>> >>>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com>
>>> >>>>> wrote:
>>> >>>>>>
>>> >>>>>> Updated the following builds:
>>> >>>>>>
>>> >>>>>> * HBase-TRUNK
>>> >>>>>>
>>> >>>>>> moved from
>>> >>>>>>
>>> >>>>>> mvn clean compile findbugs:findbugs
>>> >>>>>>
>>> >>>>>> to
>>> >>>>>>
>>> >>>>>> mvn clean -DskipTests package findbugs:findbugs
>>> >>>>>>
>>> >>>>>> To work around the known issue where we can't do a bootstrap
>>> compile
>>> >>>>>> without previous install or remote SNAPSHOT artifacts. (recently
>>> triggered
>>> >>>>>> by the addition of the procedure module on master)
>>> >>>>>>
>>> >>>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
>>> >>>>>>>
>>> >>>>>>> Make findbugs and checkstyle plugins always run.
>>> >>>>>>> The default behavior only publishes result for stable builds, but
>>> >>>>>>> master is
>>> >>>>>>> not always stable and sometimes we introduce new warnings in
>>> unstable
>>> >>>>>>> builds(see builds 6310-6314).
>>> >>>>>>> And findbugs and checkstyle will not fail unless we abort the
>>> >>>>>>> building
>>> >>>>>>> process I think, so it is safe to turn it on every time.
>>> >>>>>>>
>>> >>>>>>> Thanks.
>>> >>>>>>>
>>> >>>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>>> >>>>>>>
>>> >>>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>>> >>>>>>> >
>>> >>>>>>> > Cheers
>>> >>>>>>> >
>>> >>>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com>
>>> wrote:
>>> >>>>>>> >
>>> >>>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
>>> >>>>>>> > > findbugs,
>>> >>>>>>> > > checkstyle and jacoco coverage report.
>>> >>>>>>> > > The config has been tested on
>>> >>>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly
>>> 30
>>> >>>>>>> > > times.
>>> >>>>>>> > Not
>>> >>>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
>>> >>>>>>> > > overhead
>>> >>>>>>> > of
>>> >>>>>>> > > collecting information for code coverage).
>>> >>>>>>> > > Thanks.
>>> >>>>>>> > >
>>> >>>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <apurtell@apache.org
>>> >:
>>> >>>>>>> > >
>>> >>>>>>> > > > +1, thanks a lot for improving our build hygiene.
>>> >>>>>>> > > >
>>> >>>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar
>>> >>>>>>> > > > <en...@gmail.com>
>>> >>>>>>> > > wrote:
>>> >>>>>>> > > >
>>> >>>>>>> > > > > Thanks Sean. This is great.
>>> >>>>>>> > > > >
>>> >>>>>>> > > > > Enis
>>> >>>>>>> > > > >
>>> >>>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey
>>> >>>>>>> > > > > <bu...@cloudera.com>
>>> >>>>>>> > > > wrote:
>>> >>>>>>> > > > >
>>> >>>>>>> > > > > > FYI I just finished chasing down the breakage for mvn
>>> site
>>> >>>>>>> > > > > > on all
>>> >>>>>>> > > patch
>>> >>>>>>> > > > > > builds.
>>> >>>>>>> > > > > >
>>> >>>>>>> > > > > > HBASE-13191 consolidates the few places in the
>>> test-patch
>>> >>>>>>> > > > > > code
>>> >>>>>>> > where
>>> >>>>>>> > > we
>>> >>>>>>> > > > > > hard coded MAVEN_OPTS.
>>> >>>>>>> > > > > >
>>> >>>>>>> > > > > > If you look at the PreCommit job now, we use the "set
>>> >>>>>>> > > > > > environment
>>> >>>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything
>>> >>>>>>> > > > > > else
>>> >>>>>>> > respects
>>> >>>>>>> > > > > that
>>> >>>>>>> > > > > > setting.
>>> >>>>>>> > > > > >
>>> >>>>>>> > > > > > I set the initial value to be a combination of the
>>> memory
>>> >>>>>>> > limitations
>>> >>>>>>> > > > > we've
>>> >>>>>>> > > > > > been actually running with (the ~6G was getting ignored)
>>> >>>>>>> > > > > > and the
>>> >>>>>>> > > > permgen
>>> >>>>>>> > > > > > needed for site.
>>> >>>>>>> > > > > >
>>> >>>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData
>>> -XX:MaxPermSize=256m
>>> >>>>>>> > > > > >
>>> >>>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <
>>> stack@duboce.net>
>>> >>>>>>> > > > > > wrote:
>>> >>>>>>> > > > > >
>>> >>>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
>>> >>>>>>> > > > > > > patch-build
>>> >>>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS
>>> to
>>> >>>>>>> > > > > > > be the
>>> >>>>>>> > > same
>>> >>>>>>> > > > as
>>> >>>>>>> > > > > > > those of trunk build too, setting
>>> >>>>>>> > > > > > > MAVEN_OPTS="-Xmx6100m"... it
>>> >>>>>>> > had
>>> >>>>>>> > > > been
>>> >>>>>>> > > > > > > 3000.
>>> >>>>>>> > > > > > >
>>> >>>>>>> > > > > > > Yours,
>>> >>>>>>> > > > > > > St.Ack
>>> >>>>>>> > > > > > >
>>> >>>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <
>>> stack@duboce.net>
>>> >>>>>>> > > > > > > wrote:
>>> >>>>>>> > > > > > >
>>> >>>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds
>>> and
>>> >>>>>>> > > > > > > > or last
>>> >>>>>>> > 7
>>> >>>>>>> > > > > days,
>>> >>>>>>> > > > > > > > whichever comes first.
>>> >>>>>>> > > > > > > > St.Ack
>>> >>>>>>> > > > > > > >
>>> >>>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack
>>> >>>>>>> > > > > > > > <st...@duboce.net>
>>> >>>>>>> > wrote:
>>> >>>>>>> > > > > > > >
>>> >>>>>>> > > > > > > >> Branch-1 and master have stabilized and now run
>>> mostly
>>> >>>>>>> > > > > > > >> blue
>>> >>>>>>> > > (give
>>> >>>>>>> > > > or
>>> >>>>>>> > > > > > > take
>>> >>>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue
>>> branch-1
>>> >>>>>>> > > > > > > >> has
>>> >>>>>>> > > helped
>>> >>>>>>> > > > us
>>> >>>>>>> > > > > > > >> identify at least one destabilizing commit in the
>>> last
>>> >>>>>>> > > > > > > >> few
>>> >>>>>>> > days,
>>> >>>>>>> > > > > maybe
>>> >>>>>>> > > > > > > two;
>>> >>>>>>> > > > > > > >> this is as it should be (smile).
>>> >>>>>>> > > > > > > >>
>>> >>>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch,
>>> make
>>> >>>>>>> > > > > > > >> sure
>>> >>>>>>> > > > > subsequent
>>> >>>>>>> > > > > > > >> builds stay blue. You can subscribe to
>>> >>>>>>> > builds@hbase.apache.org
>>> >>>>>>> > > to
>>> >>>>>>> > > > > get
>>> >>>>>>> > > > > > > >> notice of failures if not already subscribed.
>>> >>>>>>> > > > > > > >>
>>> >>>>>>> > > > > > > >> Thanks,
>>> >>>>>>> > > > > > > >> St.Ack
>>> >>>>>>> > > > > > > >>
>>> >>>>>>> > > > > > > >> 1.
>>> >>>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>>> >>>>>>> > > > > > > >> 2.
>>> >>>>>>> > >
>>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>>> >>>>>>> > > > > > > >>
>>> >>>>>>> > > > > > > >>
>>> >>>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack
>>> >>>>>>> > > > > > > >> <st...@duboce.net>
>>> >>>>>>> > > wrote:
>>> >>>>>>> > > > > > > >>
>>> >>>>>>> > > > > > > >>> A few notes on testing.
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>> Too long to read, infra is more capable now and
>>> after
>>> >>>>>>> > > > > > > >>> some
>>> >>>>>>> > > work,
>>> >>>>>>> > > > we
>>> >>>>>>> > > > > > are
>>> >>>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue.
>>> Lets
>>> >>>>>>> > > > > > > >>> try and
>>> >>>>>>> > > keep
>>> >>>>>>> > > > it
>>> >>>>>>> > > > > > > this
>>> >>>>>>> > > > > > > >>> way going forward.
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>> A recent spurt of test fixing combined with more
>>> >>>>>>> > > > > > > >>> capable
>>> >>>>>>> > > hardware
>>> >>>>>>> > > > > > seems
>>> >>>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
>>> >>>>>>> > > > > > > >>> passing
>>> >>>>>>> > now
>>> >>>>>>> > > on
>>> >>>>>>> > > > > > > branch-1
>>> >>>>>>> > > > > > > >>> and master. Lets try and keep it this way and
>>> start
>>> >>>>>>> > > > > > > >>> to trust
>>> >>>>>>> > > our
>>> >>>>>>> > > > > > test
>>> >>>>>>> > > > > > > runs
>>> >>>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and
>>> nail
>>> >>>>>>> > > > > > > >>> them.
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>> Our tests now run in parallel with other test
>>> suites
>>> >>>>>>> > > > > > > >>> where
>>> >>>>>>> > > > previous
>>> >>>>>>> > > > > > we
>>> >>>>>>> > > > > > > >>> ran alone. You can see this sometimes when our
>>> zombie
>>> >>>>>>> > detector
>>> >>>>>>> > > > > > reports
>>> >>>>>>> > > > > > > >>> tests from another project altogether as lingerers
>>> >>>>>>> > > > > > > >>> (To be
>>> >>>>>>> > > fixed).
>>> >>>>>>> > > > > > > Some of
>>> >>>>>>> > > > > > > >>> our tests are failing because a concurrent hbase
>>> run
>>> >>>>>>> > > > > > > >>> is
>>> >>>>>>> > undoing
>>> >>>>>>> > > > > > > classes and
>>> >>>>>>> > > > > > > >>> data from under it. Also, lets fix.
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for
>>> them to
>>> >>>>>>> > complete.
>>> >>>>>>> > > > > Many
>>> >>>>>>> > > > > > > >>> are heavy-duty integration tests starting up
>>> multiple
>>> >>>>>>> > clusters
>>> >>>>>>> > > > and
>>> >>>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they
>>> >>>>>>> > > > > > > >>> pass at
>>> >>>>>>> > all.
>>> >>>>>>> > > > > > > Usually
>>> >>>>>>> > > > > > > >>> integration tests have been cast as unit tests
>>> >>>>>>> > > > > > > >>> because there
>>> >>>>>>> > > was
>>> >>>>>>> > > > no
>>> >>>>>>> > > > > > > where
>>> >>>>>>> > > > > > > >>> else for them to get an airing. We have the
>>> hbase-it
>>> >>>>>>> > > > > > > >>> suite
>>> >>>>>>> > now
>>> >>>>>>> > > > > which
>>> >>>>>>> > > > > > > would
>>> >>>>>>> > > > > > > >>> be a more apt place but until these are run on a
>>> >>>>>>> > > > > > > >>> regular
>>> >>>>>>> > basis
>>> >>>>>>> > > in
>>> >>>>>>> > > > > > > public
>>> >>>>>>> > > > > > > >>> for all to see, the fat integration tests
>>> disguised
>>> >>>>>>> > > > > > > >>> as unit
>>> >>>>>>> > > tests
>>> >>>>>>> > > > > > will
>>> >>>>>>> > > > > > > >>> remain. A review of our current unit tests
>>> weeding
>>> >>>>>>> > > > > > > >>> the old
>>> >>>>>>> > > cruft
>>> >>>>>>> > > > > and
>>> >>>>>>> > > > > > > the
>>> >>>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
>>> >>>>>>> > > > > > > >>> undertaking
>>> >>>>>>> > if
>>> >>>>>>> > > > > > > someone is
>>> >>>>>>> > > > > > > >>> looking to contribute.
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>> Alex Newman has been working on making our tests
>>> work
>>> >>>>>>> > > > > > > >>> up on
>>> >>>>>>> > > > travis
>>> >>>>>>> > > > > > and
>>> >>>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes
>>> end-to-end.
>>> >>>>>>> > > > > > > >>> He
>>> >>>>>>> > also
>>> >>>>>>> > > > > added
>>> >>>>>>> > > > > > in
>>> >>>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
>>> >>>>>>> > > > > > > >>> mapreduce --
>>> >>>>>>> > > > > alongside
>>> >>>>>>> > > > > > > our
>>> >>>>>>> > > > > > > >>> old "sizing" categorizations of
>>> small/medium/large.
>>> >>>>>>> > > > > > > >>> His
>>> >>>>>>> > > thinking
>>> >>>>>>> > > > > is
>>> >>>>>>> > > > > > > that
>>> >>>>>>> > > > > > > >>> we can run these categorizations in parallel so we
>>> >>>>>>> > > > > > > >>> could run
>>> >>>>>>> > > the
>>> >>>>>>> > > > > > total
>>> >>>>>>> > > > > > > >>> suite in about the time of the longest test, say
>>> >>>>>>> > 20-30minutes?
>>> >>>>>>> > > > We
>>> >>>>>>> > > > > > > could
>>> >>>>>>> > > > > > > >>> even change Apache to run them this way.
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>> FYI,
>>> >>>>>>> > > > > > > >>> St.Ack
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>>
>>> >>>>>>> > > > > > > >>
>>> >>>>>>> > > > > > > >
>>> >>>>>>> > > > > > >
>>> >>>>>>> > > > > >
>>> >>>>>>> > > > > >
>>> >>>>>>> > > > > >
>>> >>>>>>> > > > > > --
>>> >>>>>>> > > > > > Sean
>>> >>>>>>> > > > > >
>>> >>>>>>> > > > >
>>> >>>>>>> > > >
>>> >>>>>>> > > >
>>> >>>>>>> > > >
>>> >>>>>>> > > > --
>>> >>>>>>> > > > Best regards,
>>> >>>>>>> > > >
>>> >>>>>>> > > > - Andy
>>> >>>>>>> > > >
>>> >>>>>>> > > > Problems worthy of attack prove their worth by hitting
>>> back. -
>>> >>>>>>> > > > Piet
>>> >>>>>>> > Hein
>>> >>>>>>> > > > (via Tom White)
>>> >>>>>>> > > >
>>> >>>>>>> > >
>>> >>>>>>> >
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> Sean
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> Sean
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> Sean
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Sean
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Sean
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Sean
>>>
>>>
>>>
>>> --
>>> Sean
>>>
>>
>>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
Stuff seems to be basically working. I just committed a fix for this
message so it should be gone now:
grep: unknown devices method
grep: write error: Broken pipe
Hopefully we've seen last of the errant surefire kills.
We have an HBASE_TRUNK build and we have an HBASE_TRUNK_MATRIX. We should
get rid of the former now?
St.Ack
On Mon, Nov 2, 2015 at 11:03 PM, Stack <st...@duboce.net> wrote:
> I just did an edit of all of our jenkins configuration. I changed the
> post-build zombie section. The surefire-killer was hanging out here it
> seems (For detail, see HBASE-14589 Looking for the surefire-killer; builds
> being killed...). I may have messed up builds. Will fix in morning if so
> (currently builds seem a bit backed up so won't wait around).
>
> St.Ack
>
> On Fri, Oct 30, 2015 at 9:18 AM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> Hi Folks,
>>
>> Brief summary of cleanup/changes in our builds. ref thread '[DISCUSS]
>> reducing our load on builds.a.o' on reasoning or if you have concerns.
>>
>> Our project job view now has a brief summary of what's going on:
>>
>> https://builds.apache.org/view/H-L/view/HBase/
>>
>> * Where we had matrix builds, I confirmed they're doing the same thing
>> (in at least one test-configuration) as the non-matrix version
>> * I disabled jobs that were duplicating work done in matrix jobs, or
>> that only worked on a tag
>> * Made sure all non-disabled 0.98+ jobs are set to notify jira +
>> builds mailing list
>> * I updated pre-1.1 jobs to look for changes 1/day
>> * I updated 1.1 jobs to look for changes every 8 hours
>> * I updated 1.2, 1.3, and trunk to look for changes every 4 hours
>>
>> There's still some non-pressing clean up work to do, so I'll file a
>> jira later today just to make sure it's all recorded somewhere.
>>
>> On Mon, Jun 15, 2015 at 11:55 PM, Sean Busbey <bu...@cloudera.com>
>> wrote:
>> > Added new HBase-1.3 build or branch-1 and changed the existing 1.2
>> builds to
>> > use branch-1.2. (ref HBASE-13912)
>> >
>> > On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com>
>> wrote:
>> >>
>> >> I've add a new build test to run the ITs under sunny day conditions
>> using
>> >> minicluster for both java 7 and java 8 on the 1.2 line.
>> >>
>> >> https://builds.apache.org/job/HBase-1.2-IT/
>> >>
>> >> I haven't turned on notifications yet, because BulkIngest and
>> TestIngest
>> >> are read. details on HBASE-13750
>> >>
>> >> On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com>
>> wrote:
>> >>>
>> >>> sigh. that should have been "is now a matrix build".
>> >>>
>> >>> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com>
>> wrote:
>> >>>>
>> >>>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8
>> in
>> >>>> parallel.
>> >>>>
>> >>>> I saved the old job for now and left it disabled[2].
>> >>>>
>> >>>>
>> >>>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
>> >>>> [2]:
>> >>>>
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>> >>>>
>> >>>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> Sorry for the noise. I also updated the checkstyle step on
>> HBase-TRUNK
>> >>>>>
>> >>>>> from
>> >>>>> mvn checkstyle:checkstyle
>> >>>>> to
>> >>>>> mvn -DskipTests package checkstyle:checkstyle
>> >>>>>
>> >>>>> to deal with the same issue. Looks all clear now.
>> >>>>>
>> >>>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> Updated the following builds:
>> >>>>>>
>> >>>>>> * HBase-TRUNK
>> >>>>>>
>> >>>>>> moved from
>> >>>>>>
>> >>>>>> mvn clean compile findbugs:findbugs
>> >>>>>>
>> >>>>>> to
>> >>>>>>
>> >>>>>> mvn clean -DskipTests package findbugs:findbugs
>> >>>>>>
>> >>>>>> To work around the known issue where we can't do a bootstrap
>> compile
>> >>>>>> without previous install or remote SNAPSHOT artifacts. (recently
>> triggered
>> >>>>>> by the addition of the procedure module on master)
>> >>>>>>
>> >>>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
>> >>>>>>>
>> >>>>>>> Make findbugs and checkstyle plugins always run.
>> >>>>>>> The default behavior only publishes result for stable builds, but
>> >>>>>>> master is
>> >>>>>>> not always stable and sometimes we introduce new warnings in
>> unstable
>> >>>>>>> builds(see builds 6310-6314).
>> >>>>>>> And findbugs and checkstyle will not fail unless we abort the
>> >>>>>>> building
>> >>>>>>> process I think, so it is safe to turn it on every time.
>> >>>>>>>
>> >>>>>>> Thanks.
>> >>>>>>>
>> >>>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>> >>>>>>>
>> >>>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>> >>>>>>> >
>> >>>>>>> > Cheers
>> >>>>>>> >
>> >>>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com>
>> wrote:
>> >>>>>>> >
>> >>>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
>> >>>>>>> > > findbugs,
>> >>>>>>> > > checkstyle and jacoco coverage report.
>> >>>>>>> > > The config has been tested on
>> >>>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly
>> 30
>> >>>>>>> > > times.
>> >>>>>>> > Not
>> >>>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
>> >>>>>>> > > overhead
>> >>>>>>> > of
>> >>>>>>> > > collecting information for code coverage).
>> >>>>>>> > > Thanks.
>> >>>>>>> > >
>> >>>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <apurtell@apache.org
>> >:
>> >>>>>>> > >
>> >>>>>>> > > > +1, thanks a lot for improving our build hygiene.
>> >>>>>>> > > >
>> >>>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar
>> >>>>>>> > > > <en...@gmail.com>
>> >>>>>>> > > wrote:
>> >>>>>>> > > >
>> >>>>>>> > > > > Thanks Sean. This is great.
>> >>>>>>> > > > >
>> >>>>>>> > > > > Enis
>> >>>>>>> > > > >
>> >>>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey
>> >>>>>>> > > > > <bu...@cloudera.com>
>> >>>>>>> > > > wrote:
>> >>>>>>> > > > >
>> >>>>>>> > > > > > FYI I just finished chasing down the breakage for mvn
>> site
>> >>>>>>> > > > > > on all
>> >>>>>>> > > patch
>> >>>>>>> > > > > > builds.
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > HBASE-13191 consolidates the few places in the
>> test-patch
>> >>>>>>> > > > > > code
>> >>>>>>> > where
>> >>>>>>> > > we
>> >>>>>>> > > > > > hard coded MAVEN_OPTS.
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > If you look at the PreCommit job now, we use the "set
>> >>>>>>> > > > > > environment
>> >>>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything
>> >>>>>>> > > > > > else
>> >>>>>>> > respects
>> >>>>>>> > > > > that
>> >>>>>>> > > > > > setting.
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > I set the initial value to be a combination of the
>> memory
>> >>>>>>> > limitations
>> >>>>>>> > > > > we've
>> >>>>>>> > > > > > been actually running with (the ~6G was getting ignored)
>> >>>>>>> > > > > > and the
>> >>>>>>> > > > permgen
>> >>>>>>> > > > > > needed for site.
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData
>> -XX:MaxPermSize=256m
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <
>> stack@duboce.net>
>> >>>>>>> > > > > > wrote:
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
>> >>>>>>> > > > > > > patch-build
>> >>>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS
>> to
>> >>>>>>> > > > > > > be the
>> >>>>>>> > > same
>> >>>>>>> > > > as
>> >>>>>>> > > > > > > those of trunk build too, setting
>> >>>>>>> > > > > > > MAVEN_OPTS="-Xmx6100m"... it
>> >>>>>>> > had
>> >>>>>>> > > > been
>> >>>>>>> > > > > > > 3000.
>> >>>>>>> > > > > > >
>> >>>>>>> > > > > > > Yours,
>> >>>>>>> > > > > > > St.Ack
>> >>>>>>> > > > > > >
>> >>>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <
>> stack@duboce.net>
>> >>>>>>> > > > > > > wrote:
>> >>>>>>> > > > > > >
>> >>>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds
>> and
>> >>>>>>> > > > > > > > or last
>> >>>>>>> > 7
>> >>>>>>> > > > > days,
>> >>>>>>> > > > > > > > whichever comes first.
>> >>>>>>> > > > > > > > St.Ack
>> >>>>>>> > > > > > > >
>> >>>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack
>> >>>>>>> > > > > > > > <st...@duboce.net>
>> >>>>>>> > wrote:
>> >>>>>>> > > > > > > >
>> >>>>>>> > > > > > > >> Branch-1 and master have stabilized and now run
>> mostly
>> >>>>>>> > > > > > > >> blue
>> >>>>>>> > > (give
>> >>>>>>> > > > or
>> >>>>>>> > > > > > > take
>> >>>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue
>> branch-1
>> >>>>>>> > > > > > > >> has
>> >>>>>>> > > helped
>> >>>>>>> > > > us
>> >>>>>>> > > > > > > >> identify at least one destabilizing commit in the
>> last
>> >>>>>>> > > > > > > >> few
>> >>>>>>> > days,
>> >>>>>>> > > > > maybe
>> >>>>>>> > > > > > > two;
>> >>>>>>> > > > > > > >> this is as it should be (smile).
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch,
>> make
>> >>>>>>> > > > > > > >> sure
>> >>>>>>> > > > > subsequent
>> >>>>>>> > > > > > > >> builds stay blue. You can subscribe to
>> >>>>>>> > builds@hbase.apache.org
>> >>>>>>> > > to
>> >>>>>>> > > > > get
>> >>>>>>> > > > > > > >> notice of failures if not already subscribed.
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >> Thanks,
>> >>>>>>> > > > > > > >> St.Ack
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >> 1.
>> >>>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>> >>>>>>> > > > > > > >> 2.
>> >>>>>>> > >
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack
>> >>>>>>> > > > > > > >> <st...@duboce.net>
>> >>>>>>> > > wrote:
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >>> A few notes on testing.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> Too long to read, infra is more capable now and
>> after
>> >>>>>>> > > > > > > >>> some
>> >>>>>>> > > work,
>> >>>>>>> > > > we
>> >>>>>>> > > > > > are
>> >>>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue.
>> Lets
>> >>>>>>> > > > > > > >>> try and
>> >>>>>>> > > keep
>> >>>>>>> > > > it
>> >>>>>>> > > > > > > this
>> >>>>>>> > > > > > > >>> way going forward.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> A recent spurt of test fixing combined with more
>> >>>>>>> > > > > > > >>> capable
>> >>>>>>> > > hardware
>> >>>>>>> > > > > > seems
>> >>>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
>> >>>>>>> > > > > > > >>> passing
>> >>>>>>> > now
>> >>>>>>> > > on
>> >>>>>>> > > > > > > branch-1
>> >>>>>>> > > > > > > >>> and master. Lets try and keep it this way and
>> start
>> >>>>>>> > > > > > > >>> to trust
>> >>>>>>> > > our
>> >>>>>>> > > > > > test
>> >>>>>>> > > > > > > runs
>> >>>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and
>> nail
>> >>>>>>> > > > > > > >>> them.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> Our tests now run in parallel with other test
>> suites
>> >>>>>>> > > > > > > >>> where
>> >>>>>>> > > > previous
>> >>>>>>> > > > > > we
>> >>>>>>> > > > > > > >>> ran alone. You can see this sometimes when our
>> zombie
>> >>>>>>> > detector
>> >>>>>>> > > > > > reports
>> >>>>>>> > > > > > > >>> tests from another project altogether as lingerers
>> >>>>>>> > > > > > > >>> (To be
>> >>>>>>> > > fixed).
>> >>>>>>> > > > > > > Some of
>> >>>>>>> > > > > > > >>> our tests are failing because a concurrent hbase
>> run
>> >>>>>>> > > > > > > >>> is
>> >>>>>>> > undoing
>> >>>>>>> > > > > > > classes and
>> >>>>>>> > > > > > > >>> data from under it. Also, lets fix.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for
>> them to
>> >>>>>>> > complete.
>> >>>>>>> > > > > Many
>> >>>>>>> > > > > > > >>> are heavy-duty integration tests starting up
>> multiple
>> >>>>>>> > clusters
>> >>>>>>> > > > and
>> >>>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they
>> >>>>>>> > > > > > > >>> pass at
>> >>>>>>> > all.
>> >>>>>>> > > > > > > Usually
>> >>>>>>> > > > > > > >>> integration tests have been cast as unit tests
>> >>>>>>> > > > > > > >>> because there
>> >>>>>>> > > was
>> >>>>>>> > > > no
>> >>>>>>> > > > > > > where
>> >>>>>>> > > > > > > >>> else for them to get an airing. We have the
>> hbase-it
>> >>>>>>> > > > > > > >>> suite
>> >>>>>>> > now
>> >>>>>>> > > > > which
>> >>>>>>> > > > > > > would
>> >>>>>>> > > > > > > >>> be a more apt place but until these are run on a
>> >>>>>>> > > > > > > >>> regular
>> >>>>>>> > basis
>> >>>>>>> > > in
>> >>>>>>> > > > > > > public
>> >>>>>>> > > > > > > >>> for all to see, the fat integration tests
>> disguised
>> >>>>>>> > > > > > > >>> as unit
>> >>>>>>> > > tests
>> >>>>>>> > > > > > will
>> >>>>>>> > > > > > > >>> remain. A review of our current unit tests
>> weeding
>> >>>>>>> > > > > > > >>> the old
>> >>>>>>> > > cruft
>> >>>>>>> > > > > and
>> >>>>>>> > > > > > > the
>> >>>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
>> >>>>>>> > > > > > > >>> undertaking
>> >>>>>>> > if
>> >>>>>>> > > > > > > someone is
>> >>>>>>> > > > > > > >>> looking to contribute.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> Alex Newman has been working on making our tests
>> work
>> >>>>>>> > > > > > > >>> up on
>> >>>>>>> > > > travis
>> >>>>>>> > > > > > and
>> >>>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes
>> end-to-end.
>> >>>>>>> > > > > > > >>> He
>> >>>>>>> > also
>> >>>>>>> > > > > added
>> >>>>>>> > > > > > in
>> >>>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
>> >>>>>>> > > > > > > >>> mapreduce --
>> >>>>>>> > > > > alongside
>> >>>>>>> > > > > > > our
>> >>>>>>> > > > > > > >>> old "sizing" categorizations of
>> small/medium/large.
>> >>>>>>> > > > > > > >>> His
>> >>>>>>> > > thinking
>> >>>>>>> > > > > is
>> >>>>>>> > > > > > > that
>> >>>>>>> > > > > > > >>> we can run these categorizations in parallel so we
>> >>>>>>> > > > > > > >>> could run
>> >>>>>>> > > the
>> >>>>>>> > > > > > total
>> >>>>>>> > > > > > > >>> suite in about the time of the longest test, say
>> >>>>>>> > 20-30minutes?
>> >>>>>>> > > > We
>> >>>>>>> > > > > > > could
>> >>>>>>> > > > > > > >>> even change Apache to run them this way.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> FYI,
>> >>>>>>> > > > > > > >>> St.Ack
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >
>> >>>>>>> > > > > > >
>> >>>>>>> > > > > >
>> >>>>>>> > > > > >
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > --
>> >>>>>>> > > > > > Sean
>> >>>>>>> > > > > >
>> >>>>>>> > > > >
>> >>>>>>> > > >
>> >>>>>>> > > >
>> >>>>>>> > > >
>> >>>>>>> > > > --
>> >>>>>>> > > > Best regards,
>> >>>>>>> > > >
>> >>>>>>> > > > - Andy
>> >>>>>>> > > >
>> >>>>>>> > > > Problems worthy of attack prove their worth by hitting
>> back. -
>> >>>>>>> > > > Piet
>> >>>>>>> > Hein
>> >>>>>>> > > > (via Tom White)
>> >>>>>>> > > >
>> >>>>>>> > >
>> >>>>>>> >
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Sean
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Sean
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Sean
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Sean
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Sean
>> >
>> >
>> >
>> >
>> > --
>> > Sean
>>
>>
>>
>> --
>> Sean
>>
>
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
I just did an edit of all of our jenkins configuration. I changed the
post-build zombie section. The surefire-killer was hanging out here it
seems (For detail, see HBASE-14589 Looking for the surefire-killer; builds
being killed...). I may have messed up builds. Will fix in morning if so
(currently builds seem a bit backed up so won't wait around).
St.Ack
On Fri, Oct 30, 2015 at 9:18 AM, Sean Busbey <bu...@cloudera.com> wrote:
> Hi Folks,
>
> Brief summary of cleanup/changes in our builds. ref thread '[DISCUSS]
> reducing our load on builds.a.o' on reasoning or if you have concerns.
>
> Our project job view now has a brief summary of what's going on:
>
> https://builds.apache.org/view/H-L/view/HBase/
>
> * Where we had matrix builds, I confirmed they're doing the same thing
> (in at least one test-configuration) as the non-matrix version
> * I disabled jobs that were duplicating work done in matrix jobs, or
> that only worked on a tag
> * Made sure all non-disabled 0.98+ jobs are set to notify jira +
> builds mailing list
> * I updated pre-1.1 jobs to look for changes 1/day
> * I updated 1.1 jobs to look for changes every 8 hours
> * I updated 1.2, 1.3, and trunk to look for changes every 4 hours
>
> There's still some non-pressing clean up work to do, so I'll file a
> jira later today just to make sure it's all recorded somewhere.
>
> On Mon, Jun 15, 2015 at 11:55 PM, Sean Busbey <bu...@cloudera.com> wrote:
> > Added new HBase-1.3 build or branch-1 and changed the existing 1.2
> builds to
> > use branch-1.2. (ref HBASE-13912)
> >
> > On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >>
> >> I've add a new build test to run the ITs under sunny day conditions
> using
> >> minicluster for both java 7 and java 8 on the 1.2 line.
> >>
> >> https://builds.apache.org/job/HBase-1.2-IT/
> >>
> >> I haven't turned on notifications yet, because BulkIngest and TestIngest
> >> are read. details on HBASE-13750
> >>
> >> On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >>>
> >>> sigh. that should have been "is now a matrix build".
> >>>
> >>> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >>>>
> >>>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8
> in
> >>>> parallel.
> >>>>
> >>>> I saved the old job for now and left it disabled[2].
> >>>>
> >>>>
> >>>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
> >>>> [2]:
> >>>>
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
> >>>>
> >>>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com>
> >>>> wrote:
> >>>>>
> >>>>> Sorry for the noise. I also updated the checkstyle step on
> HBase-TRUNK
> >>>>>
> >>>>> from
> >>>>> mvn checkstyle:checkstyle
> >>>>> to
> >>>>> mvn -DskipTests package checkstyle:checkstyle
> >>>>>
> >>>>> to deal with the same issue. Looks all clear now.
> >>>>>
> >>>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> Updated the following builds:
> >>>>>>
> >>>>>> * HBase-TRUNK
> >>>>>>
> >>>>>> moved from
> >>>>>>
> >>>>>> mvn clean compile findbugs:findbugs
> >>>>>>
> >>>>>> to
> >>>>>>
> >>>>>> mvn clean -DskipTests package findbugs:findbugs
> >>>>>>
> >>>>>> To work around the known issue where we can't do a bootstrap compile
> >>>>>> without previous install or remote SNAPSHOT artifacts. (recently
> triggered
> >>>>>> by the addition of the procedure module on master)
> >>>>>>
> >>>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> Make findbugs and checkstyle plugins always run.
> >>>>>>> The default behavior only publishes result for stable builds, but
> >>>>>>> master is
> >>>>>>> not always stable and sometimes we introduce new warnings in
> unstable
> >>>>>>> builds(see builds 6310-6314).
> >>>>>>> And findbugs and checkstyle will not fail unless we abort the
> >>>>>>> building
> >>>>>>> process I think, so it is safe to turn it on every time.
> >>>>>>>
> >>>>>>> Thanks.
> >>>>>>>
> >>>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
> >>>>>>>
> >>>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
> >>>>>>> >
> >>>>>>> > Cheers
> >>>>>>> >
> >>>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com>
> wrote:
> >>>>>>> >
> >>>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
> >>>>>>> > > findbugs,
> >>>>>>> > > checkstyle and jacoco coverage report.
> >>>>>>> > > The config has been tested on
> >>>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30
> >>>>>>> > > times.
> >>>>>>> > Not
> >>>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
> >>>>>>> > > overhead
> >>>>>>> > of
> >>>>>>> > > collecting information for code coverage).
> >>>>>>> > > Thanks.
> >>>>>>> > >
> >>>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <apurtell@apache.org
> >:
> >>>>>>> > >
> >>>>>>> > > > +1, thanks a lot for improving our build hygiene.
> >>>>>>> > > >
> >>>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar
> >>>>>>> > > > <en...@gmail.com>
> >>>>>>> > > wrote:
> >>>>>>> > > >
> >>>>>>> > > > > Thanks Sean. This is great.
> >>>>>>> > > > >
> >>>>>>> > > > > Enis
> >>>>>>> > > > >
> >>>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey
> >>>>>>> > > > > <bu...@cloudera.com>
> >>>>>>> > > > wrote:
> >>>>>>> > > > >
> >>>>>>> > > > > > FYI I just finished chasing down the breakage for mvn
> site
> >>>>>>> > > > > > on all
> >>>>>>> > > patch
> >>>>>>> > > > > > builds.
> >>>>>>> > > > > >
> >>>>>>> > > > > > HBASE-13191 consolidates the few places in the test-patch
> >>>>>>> > > > > > code
> >>>>>>> > where
> >>>>>>> > > we
> >>>>>>> > > > > > hard coded MAVEN_OPTS.
> >>>>>>> > > > > >
> >>>>>>> > > > > > If you look at the PreCommit job now, we use the "set
> >>>>>>> > > > > > environment
> >>>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything
> >>>>>>> > > > > > else
> >>>>>>> > respects
> >>>>>>> > > > > that
> >>>>>>> > > > > > setting.
> >>>>>>> > > > > >
> >>>>>>> > > > > > I set the initial value to be a combination of the memory
> >>>>>>> > limitations
> >>>>>>> > > > > we've
> >>>>>>> > > > > > been actually running with (the ~6G was getting ignored)
> >>>>>>> > > > > > and the
> >>>>>>> > > > permgen
> >>>>>>> > > > > > needed for site.
> >>>>>>> > > > > >
> >>>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData
> -XX:MaxPermSize=256m
> >>>>>>> > > > > >
> >>>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <
> stack@duboce.net>
> >>>>>>> > > > > > wrote:
> >>>>>>> > > > > >
> >>>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
> >>>>>>> > > > > > > patch-build
> >>>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS
> to
> >>>>>>> > > > > > > be the
> >>>>>>> > > same
> >>>>>>> > > > as
> >>>>>>> > > > > > > those of trunk build too, setting
> >>>>>>> > > > > > > MAVEN_OPTS="-Xmx6100m"... it
> >>>>>>> > had
> >>>>>>> > > > been
> >>>>>>> > > > > > > 3000.
> >>>>>>> > > > > > >
> >>>>>>> > > > > > > Yours,
> >>>>>>> > > > > > > St.Ack
> >>>>>>> > > > > > >
> >>>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <
> stack@duboce.net>
> >>>>>>> > > > > > > wrote:
> >>>>>>> > > > > > >
> >>>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds
> and
> >>>>>>> > > > > > > > or last
> >>>>>>> > 7
> >>>>>>> > > > > days,
> >>>>>>> > > > > > > > whichever comes first.
> >>>>>>> > > > > > > > St.Ack
> >>>>>>> > > > > > > >
> >>>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack
> >>>>>>> > > > > > > > <st...@duboce.net>
> >>>>>>> > wrote:
> >>>>>>> > > > > > > >
> >>>>>>> > > > > > > >> Branch-1 and master have stabilized and now run
> mostly
> >>>>>>> > > > > > > >> blue
> >>>>>>> > > (give
> >>>>>>> > > > or
> >>>>>>> > > > > > > take
> >>>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue
> branch-1
> >>>>>>> > > > > > > >> has
> >>>>>>> > > helped
> >>>>>>> > > > us
> >>>>>>> > > > > > > >> identify at least one destabilizing commit in the
> last
> >>>>>>> > > > > > > >> few
> >>>>>>> > days,
> >>>>>>> > > > > maybe
> >>>>>>> > > > > > > two;
> >>>>>>> > > > > > > >> this is as it should be (smile).
> >>>>>>> > > > > > > >>
> >>>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch,
> make
> >>>>>>> > > > > > > >> sure
> >>>>>>> > > > > subsequent
> >>>>>>> > > > > > > >> builds stay blue. You can subscribe to
> >>>>>>> > builds@hbase.apache.org
> >>>>>>> > > to
> >>>>>>> > > > > get
> >>>>>>> > > > > > > >> notice of failures if not already subscribed.
> >>>>>>> > > > > > > >>
> >>>>>>> > > > > > > >> Thanks,
> >>>>>>> > > > > > > >> St.Ack
> >>>>>>> > > > > > > >>
> >>>>>>> > > > > > > >> 1.
> >>>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> >>>>>>> > > > > > > >> 2.
> >>>>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> >>>>>>> > > > > > > >>
> >>>>>>> > > > > > > >>
> >>>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack
> >>>>>>> > > > > > > >> <st...@duboce.net>
> >>>>>>> > > wrote:
> >>>>>>> > > > > > > >>
> >>>>>>> > > > > > > >>> A few notes on testing.
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>> Too long to read, infra is more capable now and
> after
> >>>>>>> > > > > > > >>> some
> >>>>>>> > > work,
> >>>>>>> > > > we
> >>>>>>> > > > > > are
> >>>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets
> >>>>>>> > > > > > > >>> try and
> >>>>>>> > > keep
> >>>>>>> > > > it
> >>>>>>> > > > > > > this
> >>>>>>> > > > > > > >>> way going forward.
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>> A recent spurt of test fixing combined with more
> >>>>>>> > > > > > > >>> capable
> >>>>>>> > > hardware
> >>>>>>> > > > > > seems
> >>>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
> >>>>>>> > > > > > > >>> passing
> >>>>>>> > now
> >>>>>>> > > on
> >>>>>>> > > > > > > branch-1
> >>>>>>> > > > > > > >>> and master. Lets try and keep it this way and
> start
> >>>>>>> > > > > > > >>> to trust
> >>>>>>> > > our
> >>>>>>> > > > > > test
> >>>>>>> > > > > > > runs
> >>>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and
> nail
> >>>>>>> > > > > > > >>> them.
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>> Our tests now run in parallel with other test
> suites
> >>>>>>> > > > > > > >>> where
> >>>>>>> > > > previous
> >>>>>>> > > > > > we
> >>>>>>> > > > > > > >>> ran alone. You can see this sometimes when our
> zombie
> >>>>>>> > detector
> >>>>>>> > > > > > reports
> >>>>>>> > > > > > > >>> tests from another project altogether as lingerers
> >>>>>>> > > > > > > >>> (To be
> >>>>>>> > > fixed).
> >>>>>>> > > > > > > Some of
> >>>>>>> > > > > > > >>> our tests are failing because a concurrent hbase
> run
> >>>>>>> > > > > > > >>> is
> >>>>>>> > undoing
> >>>>>>> > > > > > > classes and
> >>>>>>> > > > > > > >>> data from under it. Also, lets fix.
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them
> to
> >>>>>>> > complete.
> >>>>>>> > > > > Many
> >>>>>>> > > > > > > >>> are heavy-duty integration tests starting up
> multiple
> >>>>>>> > clusters
> >>>>>>> > > > and
> >>>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they
> >>>>>>> > > > > > > >>> pass at
> >>>>>>> > all.
> >>>>>>> > > > > > > Usually
> >>>>>>> > > > > > > >>> integration tests have been cast as unit tests
> >>>>>>> > > > > > > >>> because there
> >>>>>>> > > was
> >>>>>>> > > > no
> >>>>>>> > > > > > > where
> >>>>>>> > > > > > > >>> else for them to get an airing. We have the
> hbase-it
> >>>>>>> > > > > > > >>> suite
> >>>>>>> > now
> >>>>>>> > > > > which
> >>>>>>> > > > > > > would
> >>>>>>> > > > > > > >>> be a more apt place but until these are run on a
> >>>>>>> > > > > > > >>> regular
> >>>>>>> > basis
> >>>>>>> > > in
> >>>>>>> > > > > > > public
> >>>>>>> > > > > > > >>> for all to see, the fat integration tests disguised
> >>>>>>> > > > > > > >>> as unit
> >>>>>>> > > tests
> >>>>>>> > > > > > will
> >>>>>>> > > > > > > >>> remain. A review of our current unit tests weeding
> >>>>>>> > > > > > > >>> the old
> >>>>>>> > > cruft
> >>>>>>> > > > > and
> >>>>>>> > > > > > > the
> >>>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
> >>>>>>> > > > > > > >>> undertaking
> >>>>>>> > if
> >>>>>>> > > > > > > someone is
> >>>>>>> > > > > > > >>> looking to contribute.
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>> Alex Newman has been working on making our tests
> work
> >>>>>>> > > > > > > >>> up on
> >>>>>>> > > > travis
> >>>>>>> > > > > > and
> >>>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes
> end-to-end.
> >>>>>>> > > > > > > >>> He
> >>>>>>> > also
> >>>>>>> > > > > added
> >>>>>>> > > > > > in
> >>>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
> >>>>>>> > > > > > > >>> mapreduce --
> >>>>>>> > > > > alongside
> >>>>>>> > > > > > > our
> >>>>>>> > > > > > > >>> old "sizing" categorizations of small/medium/large.
> >>>>>>> > > > > > > >>> His
> >>>>>>> > > thinking
> >>>>>>> > > > > is
> >>>>>>> > > > > > > that
> >>>>>>> > > > > > > >>> we can run these categorizations in parallel so we
> >>>>>>> > > > > > > >>> could run
> >>>>>>> > > the
> >>>>>>> > > > > > total
> >>>>>>> > > > > > > >>> suite in about the time of the longest test, say
> >>>>>>> > 20-30minutes?
> >>>>>>> > > > We
> >>>>>>> > > > > > > could
> >>>>>>> > > > > > > >>> even change Apache to run them this way.
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>> FYI,
> >>>>>>> > > > > > > >>> St.Ack
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>>
> >>>>>>> > > > > > > >>
> >>>>>>> > > > > > > >
> >>>>>>> > > > > > >
> >>>>>>> > > > > >
> >>>>>>> > > > > >
> >>>>>>> > > > > >
> >>>>>>> > > > > > --
> >>>>>>> > > > > > Sean
> >>>>>>> > > > > >
> >>>>>>> > > > >
> >>>>>>> > > >
> >>>>>>> > > >
> >>>>>>> > > >
> >>>>>>> > > > --
> >>>>>>> > > > Best regards,
> >>>>>>> > > >
> >>>>>>> > > > - Andy
> >>>>>>> > > >
> >>>>>>> > > > Problems worthy of attack prove their worth by hitting back.
> -
> >>>>>>> > > > Piet
> >>>>>>> > Hein
> >>>>>>> > > > (via Tom White)
> >>>>>>> > > >
> >>>>>>> > >
> >>>>>>> >
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Sean
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Sean
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Sean
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Sean
> >>
> >>
> >>
> >>
> >> --
> >> Sean
> >
> >
> >
> >
> > --
> > Sean
>
>
>
> --
> Sean
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
Hi Folks,
Brief summary of cleanup/changes in our builds. ref thread '[DISCUSS]
reducing our load on builds.a.o' on reasoning or if you have concerns.
Our project job view now has a brief summary of what's going on:
https://builds.apache.org/view/H-L/view/HBase/
* Where we had matrix builds, I confirmed they're doing the same thing
(in at least one test-configuration) as the non-matrix version
* I disabled jobs that were duplicating work done in matrix jobs, or
that only worked on a tag
* Made sure all non-disabled 0.98+ jobs are set to notify jira +
builds mailing list
* I updated pre-1.1 jobs to look for changes 1/day
* I updated 1.1 jobs to look for changes every 8 hours
* I updated 1.2, 1.3, and trunk to look for changes every 4 hours
There's still some non-pressing clean up work to do, so I'll file a
jira later today just to make sure it's all recorded somewhere.
On Mon, Jun 15, 2015 at 11:55 PM, Sean Busbey <bu...@cloudera.com> wrote:
> Added new HBase-1.3 build or branch-1 and changed the existing 1.2 builds to
> use branch-1.2. (ref HBASE-13912)
>
> On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com> wrote:
>>
>> I've add a new build test to run the ITs under sunny day conditions using
>> minicluster for both java 7 and java 8 on the 1.2 line.
>>
>> https://builds.apache.org/job/HBase-1.2-IT/
>>
>> I haven't turned on notifications yet, because BulkIngest and TestIngest
>> are read. details on HBASE-13750
>>
>> On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com> wrote:
>>>
>>> sigh. that should have been "is now a matrix build".
>>>
>>> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com> wrote:
>>>>
>>>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 in
>>>> parallel.
>>>>
>>>> I saved the old job for now and left it disabled[2].
>>>>
>>>>
>>>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
>>>> [2]:
>>>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>>>>
>>>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com>
>>>> wrote:
>>>>>
>>>>> Sorry for the noise. I also updated the checkstyle step on HBase-TRUNK
>>>>>
>>>>> from
>>>>> mvn checkstyle:checkstyle
>>>>> to
>>>>> mvn -DskipTests package checkstyle:checkstyle
>>>>>
>>>>> to deal with the same issue. Looks all clear now.
>>>>>
>>>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com>
>>>>> wrote:
>>>>>>
>>>>>> Updated the following builds:
>>>>>>
>>>>>> * HBase-TRUNK
>>>>>>
>>>>>> moved from
>>>>>>
>>>>>> mvn clean compile findbugs:findbugs
>>>>>>
>>>>>> to
>>>>>>
>>>>>> mvn clean -DskipTests package findbugs:findbugs
>>>>>>
>>>>>> To work around the known issue where we can't do a bootstrap compile
>>>>>> without previous install or remote SNAPSHOT artifacts. (recently triggered
>>>>>> by the addition of the procedure module on master)
>>>>>>
>>>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
>>>>>>>
>>>>>>> Make findbugs and checkstyle plugins always run.
>>>>>>> The default behavior only publishes result for stable builds, but
>>>>>>> master is
>>>>>>> not always stable and sometimes we introduce new warnings in unstable
>>>>>>> builds(see builds 6310-6314).
>>>>>>> And findbugs and checkstyle will not fail unless we abort the
>>>>>>> building
>>>>>>> process I think, so it is safe to turn it on every time.
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>>>>>>>
>>>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>>>>>>> >
>>>>>>> > Cheers
>>>>>>> >
>>>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com> wrote:
>>>>>>> >
>>>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
>>>>>>> > > findbugs,
>>>>>>> > > checkstyle and jacoco coverage report.
>>>>>>> > > The config has been tested on
>>>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30
>>>>>>> > > times.
>>>>>>> > Not
>>>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
>>>>>>> > > overhead
>>>>>>> > of
>>>>>>> > > collecting information for code coverage).
>>>>>>> > > Thanks.
>>>>>>> > >
>>>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>>>>>>> > >
>>>>>>> > > > +1, thanks a lot for improving our build hygiene.
>>>>>>> > > >
>>>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar
>>>>>>> > > > <en...@gmail.com>
>>>>>>> > > wrote:
>>>>>>> > > >
>>>>>>> > > > > Thanks Sean. This is great.
>>>>>>> > > > >
>>>>>>> > > > > Enis
>>>>>>> > > > >
>>>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey
>>>>>>> > > > > <bu...@cloudera.com>
>>>>>>> > > > wrote:
>>>>>>> > > > >
>>>>>>> > > > > > FYI I just finished chasing down the breakage for mvn site
>>>>>>> > > > > > on all
>>>>>>> > > patch
>>>>>>> > > > > > builds.
>>>>>>> > > > > >
>>>>>>> > > > > > HBASE-13191 consolidates the few places in the test-patch
>>>>>>> > > > > > code
>>>>>>> > where
>>>>>>> > > we
>>>>>>> > > > > > hard coded MAVEN_OPTS.
>>>>>>> > > > > >
>>>>>>> > > > > > If you look at the PreCommit job now, we use the "set
>>>>>>> > > > > > environment
>>>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything
>>>>>>> > > > > > else
>>>>>>> > respects
>>>>>>> > > > > that
>>>>>>> > > > > > setting.
>>>>>>> > > > > >
>>>>>>> > > > > > I set the initial value to be a combination of the memory
>>>>>>> > limitations
>>>>>>> > > > > we've
>>>>>>> > > > > > been actually running with (the ~6G was getting ignored)
>>>>>>> > > > > > and the
>>>>>>> > > > permgen
>>>>>>> > > > > > needed for site.
>>>>>>> > > > > >
>>>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
>>>>>>> > > > > >
>>>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net>
>>>>>>> > > > > > wrote:
>>>>>>> > > > > >
>>>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
>>>>>>> > > > > > > patch-build
>>>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to
>>>>>>> > > > > > > be the
>>>>>>> > > same
>>>>>>> > > > as
>>>>>>> > > > > > > those of trunk build too, setting
>>>>>>> > > > > > > MAVEN_OPTS="-Xmx6100m"... it
>>>>>>> > had
>>>>>>> > > > been
>>>>>>> > > > > > > 3000.
>>>>>>> > > > > > >
>>>>>>> > > > > > > Yours,
>>>>>>> > > > > > > St.Ack
>>>>>>> > > > > > >
>>>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net>
>>>>>>> > > > > > > wrote:
>>>>>>> > > > > > >
>>>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds and
>>>>>>> > > > > > > > or last
>>>>>>> > 7
>>>>>>> > > > > days,
>>>>>>> > > > > > > > whichever comes first.
>>>>>>> > > > > > > > St.Ack
>>>>>>> > > > > > > >
>>>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack
>>>>>>> > > > > > > > <st...@duboce.net>
>>>>>>> > wrote:
>>>>>>> > > > > > > >
>>>>>>> > > > > > > >> Branch-1 and master have stabilized and now run mostly
>>>>>>> > > > > > > >> blue
>>>>>>> > > (give
>>>>>>> > > > or
>>>>>>> > > > > > > take
>>>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1
>>>>>>> > > > > > > >> has
>>>>>>> > > helped
>>>>>>> > > > us
>>>>>>> > > > > > > >> identify at least one destabilizing commit in the last
>>>>>>> > > > > > > >> few
>>>>>>> > days,
>>>>>>> > > > > maybe
>>>>>>> > > > > > > two;
>>>>>>> > > > > > > >> this is as it should be (smile).
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch, make
>>>>>>> > > > > > > >> sure
>>>>>>> > > > > subsequent
>>>>>>> > > > > > > >> builds stay blue. You can subscribe to
>>>>>>> > builds@hbase.apache.org
>>>>>>> > > to
>>>>>>> > > > > get
>>>>>>> > > > > > > >> notice of failures if not already subscribed.
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >> Thanks,
>>>>>>> > > > > > > >> St.Ack
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >> 1.
>>>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>>>>>>> > > > > > > >> 2.
>>>>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack
>>>>>>> > > > > > > >> <st...@duboce.net>
>>>>>>> > > wrote:
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >>> A few notes on testing.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> Too long to read, infra is more capable now and after
>>>>>>> > > > > > > >>> some
>>>>>>> > > work,
>>>>>>> > > > we
>>>>>>> > > > > > are
>>>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets
>>>>>>> > > > > > > >>> try and
>>>>>>> > > keep
>>>>>>> > > > it
>>>>>>> > > > > > > this
>>>>>>> > > > > > > >>> way going forward.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> A recent spurt of test fixing combined with more
>>>>>>> > > > > > > >>> capable
>>>>>>> > > hardware
>>>>>>> > > > > > seems
>>>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
>>>>>>> > > > > > > >>> passing
>>>>>>> > now
>>>>>>> > > on
>>>>>>> > > > > > > branch-1
>>>>>>> > > > > > > >>> and master. Lets try and keep it this way and start
>>>>>>> > > > > > > >>> to trust
>>>>>>> > > our
>>>>>>> > > > > > test
>>>>>>> > > > > > > runs
>>>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and nail
>>>>>>> > > > > > > >>> them.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> Our tests now run in parallel with other test suites
>>>>>>> > > > > > > >>> where
>>>>>>> > > > previous
>>>>>>> > > > > > we
>>>>>>> > > > > > > >>> ran alone. You can see this sometimes when our zombie
>>>>>>> > detector
>>>>>>> > > > > > reports
>>>>>>> > > > > > > >>> tests from another project altogether as lingerers
>>>>>>> > > > > > > >>> (To be
>>>>>>> > > fixed).
>>>>>>> > > > > > > Some of
>>>>>>> > > > > > > >>> our tests are failing because a concurrent hbase run
>>>>>>> > > > > > > >>> is
>>>>>>> > undoing
>>>>>>> > > > > > > classes and
>>>>>>> > > > > > > >>> data from under it. Also, lets fix.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them to
>>>>>>> > complete.
>>>>>>> > > > > Many
>>>>>>> > > > > > > >>> are heavy-duty integration tests starting up multiple
>>>>>>> > clusters
>>>>>>> > > > and
>>>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they
>>>>>>> > > > > > > >>> pass at
>>>>>>> > all.
>>>>>>> > > > > > > Usually
>>>>>>> > > > > > > >>> integration tests have been cast as unit tests
>>>>>>> > > > > > > >>> because there
>>>>>>> > > was
>>>>>>> > > > no
>>>>>>> > > > > > > where
>>>>>>> > > > > > > >>> else for them to get an airing. We have the hbase-it
>>>>>>> > > > > > > >>> suite
>>>>>>> > now
>>>>>>> > > > > which
>>>>>>> > > > > > > would
>>>>>>> > > > > > > >>> be a more apt place but until these are run on a
>>>>>>> > > > > > > >>> regular
>>>>>>> > basis
>>>>>>> > > in
>>>>>>> > > > > > > public
>>>>>>> > > > > > > >>> for all to see, the fat integration tests disguised
>>>>>>> > > > > > > >>> as unit
>>>>>>> > > tests
>>>>>>> > > > > > will
>>>>>>> > > > > > > >>> remain. A review of our current unit tests weeding
>>>>>>> > > > > > > >>> the old
>>>>>>> > > cruft
>>>>>>> > > > > and
>>>>>>> > > > > > > the
>>>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
>>>>>>> > > > > > > >>> undertaking
>>>>>>> > if
>>>>>>> > > > > > > someone is
>>>>>>> > > > > > > >>> looking to contribute.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> Alex Newman has been working on making our tests work
>>>>>>> > > > > > > >>> up on
>>>>>>> > > > travis
>>>>>>> > > > > > and
>>>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes end-to-end.
>>>>>>> > > > > > > >>> He
>>>>>>> > also
>>>>>>> > > > > added
>>>>>>> > > > > > in
>>>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
>>>>>>> > > > > > > >>> mapreduce --
>>>>>>> > > > > alongside
>>>>>>> > > > > > > our
>>>>>>> > > > > > > >>> old "sizing" categorizations of small/medium/large.
>>>>>>> > > > > > > >>> His
>>>>>>> > > thinking
>>>>>>> > > > > is
>>>>>>> > > > > > > that
>>>>>>> > > > > > > >>> we can run these categorizations in parallel so we
>>>>>>> > > > > > > >>> could run
>>>>>>> > > the
>>>>>>> > > > > > total
>>>>>>> > > > > > > >>> suite in about the time of the longest test, say
>>>>>>> > 20-30minutes?
>>>>>>> > > > We
>>>>>>> > > > > > > could
>>>>>>> > > > > > > >>> even change Apache to run them this way.
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>> FYI,
>>>>>>> > > > > > > >>> St.Ack
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>>
>>>>>>> > > > > > > >>
>>>>>>> > > > > > > >
>>>>>>> > > > > > >
>>>>>>> > > > > >
>>>>>>> > > > > >
>>>>>>> > > > > >
>>>>>>> > > > > > --
>>>>>>> > > > > > Sean
>>>>>>> > > > > >
>>>>>>> > > > >
>>>>>>> > > >
>>>>>>> > > >
>>>>>>> > > >
>>>>>>> > > > --
>>>>>>> > > > Best regards,
>>>>>>> > > >
>>>>>>> > > > - Andy
>>>>>>> > > >
>>>>>>> > > > Problems worthy of attack prove their worth by hitting back. -
>>>>>>> > > > Piet
>>>>>>> > Hein
>>>>>>> > > > (via Tom White)
>>>>>>> > > >
>>>>>>> > >
>>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sean
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sean
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sean
>>>
>>>
>>>
>>>
>>> --
>>> Sean
>>
>>
>>
>>
>> --
>> Sean
>
>
>
>
> --
> Sean
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
Added new HBase-1.3 build or branch-1 and changed the existing 1.2 builds
to use branch-1.2. (ref HBASE-13912)
On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bu...@cloudera.com> wrote:
> I've add a new build test to run the ITs under sunny day conditions using
> minicluster for both java 7 and java 8 on the 1.2 line.
>
> https://builds.apache.org/job/HBase-1.2-IT/
>
> I haven't turned on notifications yet, because BulkIngest and TestIngest
> are read. details on HBASE-13750
>
> On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> sigh. that should have been "is now a matrix build".
>>
>> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com> wrote:
>>
>>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 in
>>> parallel.
>>>
>>> I saved the old job for now and left it disabled[2].
>>>
>>>
>>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
>>> [2]:
>>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>>>
>>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com>
>>> wrote:
>>>
>>>> Sorry for the noise. I also updated the checkstyle step on HBase-TRUNK
>>>>
>>>> from
>>>> mvn checkstyle:checkstyle
>>>> to
>>>> mvn -DskipTests package checkstyle:checkstyle
>>>>
>>>> to deal with the same issue. Looks all clear now.
>>>>
>>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com>
>>>> wrote:
>>>>
>>>>> Updated the following builds:
>>>>>
>>>>> * HBase-TRUNK
>>>>>
>>>>> moved from
>>>>>
>>>>> mvn clean compile findbugs:findbugs
>>>>>
>>>>> to
>>>>>
>>>>> mvn clean -DskipTests package findbugs:findbugs
>>>>>
>>>>> To work around the known issue where we can't do a bootstrap compile
>>>>> without previous install or remote SNAPSHOT artifacts. (recently triggered
>>>>> by the addition of the procedure module on master)
>>>>>
>>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
>>>>>
>>>>>> Make findbugs and checkstyle plugins always run.
>>>>>> The default behavior only publishes result for stable builds, but
>>>>>> master is
>>>>>> not always stable and sometimes we introduce new warnings in unstable
>>>>>> builds(see builds 6310-6314).
>>>>>> And findbugs and checkstyle will not fail unless we abort the building
>>>>>> process I think, so it is safe to turn it on every time.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>>>>>>
>>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>>>>>> >
>>>>>> > Cheers
>>>>>> >
>>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com> wrote:
>>>>>> >
>>>>>> > > Going to change the config of HBase-TRUNK jenkins to show
>>>>>> findbugs,
>>>>>> > > checkstyle and jacoco coverage report.
>>>>>> > > The config has been tested on
>>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30
>>>>>> times.
>>>>>> > Not
>>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
>>>>>> overhead
>>>>>> > of
>>>>>> > > collecting information for code coverage).
>>>>>> > > Thanks.
>>>>>> > >
>>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>>>>>> > >
>>>>>> > > > +1, thanks a lot for improving our build hygiene.
>>>>>> > > >
>>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <
>>>>>> enis.soz@gmail.com>
>>>>>> > > wrote:
>>>>>> > > >
>>>>>> > > > > Thanks Sean. This is great.
>>>>>> > > > >
>>>>>> > > > > Enis
>>>>>> > > > >
>>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <
>>>>>> busbey@cloudera.com>
>>>>>> > > > wrote:
>>>>>> > > > >
>>>>>> > > > > > FYI I just finished chasing down the breakage for mvn site
>>>>>> on all
>>>>>> > > patch
>>>>>> > > > > > builds.
>>>>>> > > > > >
>>>>>> > > > > > HBASE-13191 consolidates the few places in the test-patch
>>>>>> code
>>>>>> > where
>>>>>> > > we
>>>>>> > > > > > hard coded MAVEN_OPTS.
>>>>>> > > > > >
>>>>>> > > > > > If you look at the PreCommit job now, we use the "set
>>>>>> environment
>>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything else
>>>>>> > respects
>>>>>> > > > > that
>>>>>> > > > > > setting.
>>>>>> > > > > >
>>>>>> > > > > > I set the initial value to be a combination of the memory
>>>>>> > limitations
>>>>>> > > > > we've
>>>>>> > > > > > been actually running with (the ~6G was getting ignored)
>>>>>> and the
>>>>>> > > > permgen
>>>>>> > > > > > needed for site.
>>>>>> > > > > >
>>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
>>>>>> > > > > >
>>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net>
>>>>>> wrote:
>>>>>> > > > > >
>>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
>>>>>> patch-build
>>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to
>>>>>> be the
>>>>>> > > same
>>>>>> > > > as
>>>>>> > > > > > > those of trunk build too, setting
>>>>>> MAVEN_OPTS="-Xmx6100m"... it
>>>>>> > had
>>>>>> > > > been
>>>>>> > > > > > > 3000.
>>>>>> > > > > > >
>>>>>> > > > > > > Yours,
>>>>>> > > > > > > St.Ack
>>>>>> > > > > > >
>>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net>
>>>>>> wrote:
>>>>>> > > > > > >
>>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds and
>>>>>> or last
>>>>>> > 7
>>>>>> > > > > days,
>>>>>> > > > > > > > whichever comes first.
>>>>>> > > > > > > > St.Ack
>>>>>> > > > > > > >
>>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <stack@duboce.net
>>>>>> >
>>>>>> > wrote:
>>>>>> > > > > > > >
>>>>>> > > > > > > >> Branch-1 and master have stabilized and now run mostly
>>>>>> blue
>>>>>> > > (give
>>>>>> > > > or
>>>>>> > > > > > > take
>>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1
>>>>>> has
>>>>>> > > helped
>>>>>> > > > us
>>>>>> > > > > > > >> identify at least one destabilizing commit in the last
>>>>>> few
>>>>>> > days,
>>>>>> > > > > maybe
>>>>>> > > > > > > two;
>>>>>> > > > > > > >> this is as it should be (smile).
>>>>>> > > > > > > >>
>>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch, make
>>>>>> sure
>>>>>> > > > > subsequent
>>>>>> > > > > > > >> builds stay blue. You can subscribe to
>>>>>> > builds@hbase.apache.org
>>>>>> > > to
>>>>>> > > > > get
>>>>>> > > > > > > >> notice of failures if not already subscribed.
>>>>>> > > > > > > >>
>>>>>> > > > > > > >> Thanks,
>>>>>> > > > > > > >> St.Ack
>>>>>> > > > > > > >>
>>>>>> > > > > > > >> 1.
>>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>>>>>> > > > > > > >> 2.
>>>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>>>>>> > > > > > > >>
>>>>>> > > > > > > >>
>>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <
>>>>>> stack@duboce.net>
>>>>>> > > wrote:
>>>>>> > > > > > > >>
>>>>>> > > > > > > >>> A few notes on testing.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> Too long to read, infra is more capable now and after
>>>>>> some
>>>>>> > > work,
>>>>>> > > > we
>>>>>> > > > > > are
>>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets
>>>>>> try and
>>>>>> > > keep
>>>>>> > > > it
>>>>>> > > > > > > this
>>>>>> > > > > > > >>> way going forward.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> A recent spurt of test fixing combined with more
>>>>>> capable
>>>>>> > > hardware
>>>>>> > > > > > seems
>>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
>>>>>> passing
>>>>>> > now
>>>>>> > > on
>>>>>> > > > > > > branch-1
>>>>>> > > > > > > >>> and master. Lets try and keep it this way and start
>>>>>> to trust
>>>>>> > > our
>>>>>> > > > > > test
>>>>>> > > > > > > runs
>>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and nail
>>>>>> them.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> Our tests now run in parallel with other test suites
>>>>>> where
>>>>>> > > > previous
>>>>>> > > > > > we
>>>>>> > > > > > > >>> ran alone. You can see this sometimes when our zombie
>>>>>> > detector
>>>>>> > > > > > reports
>>>>>> > > > > > > >>> tests from another project altogether as lingerers
>>>>>> (To be
>>>>>> > > fixed).
>>>>>> > > > > > > Some of
>>>>>> > > > > > > >>> our tests are failing because a concurrent hbase run
>>>>>> is
>>>>>> > undoing
>>>>>> > > > > > > classes and
>>>>>> > > > > > > >>> data from under it. Also, lets fix.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them to
>>>>>> > complete.
>>>>>> > > > > Many
>>>>>> > > > > > > >>> are heavy-duty integration tests starting up multiple
>>>>>> > clusters
>>>>>> > > > and
>>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they
>>>>>> pass at
>>>>>> > all.
>>>>>> > > > > > > Usually
>>>>>> > > > > > > >>> integration tests have been cast as unit tests
>>>>>> because there
>>>>>> > > was
>>>>>> > > > no
>>>>>> > > > > > > where
>>>>>> > > > > > > >>> else for them to get an airing. We have the hbase-it
>>>>>> suite
>>>>>> > now
>>>>>> > > > > which
>>>>>> > > > > > > would
>>>>>> > > > > > > >>> be a more apt place but until these are run on a
>>>>>> regular
>>>>>> > basis
>>>>>> > > in
>>>>>> > > > > > > public
>>>>>> > > > > > > >>> for all to see, the fat integration tests disguised
>>>>>> as unit
>>>>>> > > tests
>>>>>> > > > > > will
>>>>>> > > > > > > >>> remain. A review of our current unit tests weeding
>>>>>> the old
>>>>>> > > cruft
>>>>>> > > > > and
>>>>>> > > > > > > the
>>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
>>>>>> undertaking
>>>>>> > if
>>>>>> > > > > > > someone is
>>>>>> > > > > > > >>> looking to contribute.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> Alex Newman has been working on making our tests work
>>>>>> up on
>>>>>> > > > travis
>>>>>> > > > > > and
>>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes
>>>>>> end-to-end. He
>>>>>> > also
>>>>>> > > > > added
>>>>>> > > > > > in
>>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
>>>>>> mapreduce --
>>>>>> > > > > alongside
>>>>>> > > > > > > our
>>>>>> > > > > > > >>> old "sizing" categorizations of small/medium/large.
>>>>>> His
>>>>>> > > thinking
>>>>>> > > > > is
>>>>>> > > > > > > that
>>>>>> > > > > > > >>> we can run these categorizations in parallel so we
>>>>>> could run
>>>>>> > > the
>>>>>> > > > > > total
>>>>>> > > > > > > >>> suite in about the time of the longest test, say
>>>>>> > 20-30minutes?
>>>>>> > > > We
>>>>>> > > > > > > could
>>>>>> > > > > > > >>> even change Apache to run them this way.
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>> FYI,
>>>>>> > > > > > > >>> St.Ack
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>>
>>>>>> > > > > > > >>
>>>>>> > > > > > > >
>>>>>> > > > > > >
>>>>>> > > > > >
>>>>>> > > > > >
>>>>>> > > > > >
>>>>>> > > > > > --
>>>>>> > > > > > Sean
>>>>>> > > > > >
>>>>>> > > > >
>>>>>> > > >
>>>>>> > > >
>>>>>> > > >
>>>>>> > > > --
>>>>>> > > > Best regards,
>>>>>> > > >
>>>>>> > > > - Andy
>>>>>> > > >
>>>>>> > > > Problems worthy of attack prove their worth by hitting back. -
>>>>>> Piet
>>>>>> > Hein
>>>>>> > > > (via Tom White)
>>>>>> > > >
>>>>>> > >
>>>>>> >
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sean
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sean
>>>>
>>>
>>>
>>>
>>> --
>>> Sean
>>>
>>
>>
>>
>> --
>> Sean
>>
>
>
>
> --
> Sean
>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
I've add a new build test to run the ITs under sunny day conditions using
minicluster for both java 7 and java 8 on the 1.2 line.
https://builds.apache.org/job/HBase-1.2-IT/
I haven't turned on notifications yet, because BulkIngest and TestIngest
are read. details on HBASE-13750
On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bu...@cloudera.com> wrote:
> sigh. that should have been "is now a matrix build".
>
> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 in
>> parallel.
>>
>> I saved the old job for now and left it disabled[2].
>>
>>
>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
>> [2]:
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>>
>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com> wrote:
>>
>>> Sorry for the noise. I also updated the checkstyle step on HBase-TRUNK
>>>
>>> from
>>> mvn checkstyle:checkstyle
>>> to
>>> mvn -DskipTests package checkstyle:checkstyle
>>>
>>> to deal with the same issue. Looks all clear now.
>>>
>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com>
>>> wrote:
>>>
>>>> Updated the following builds:
>>>>
>>>> * HBase-TRUNK
>>>>
>>>> moved from
>>>>
>>>> mvn clean compile findbugs:findbugs
>>>>
>>>> to
>>>>
>>>> mvn clean -DskipTests package findbugs:findbugs
>>>>
>>>> To work around the known issue where we can't do a bootstrap compile
>>>> without previous install or remote SNAPSHOT artifacts. (recently triggered
>>>> by the addition of the procedure module on master)
>>>>
>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
>>>>
>>>>> Make findbugs and checkstyle plugins always run.
>>>>> The default behavior only publishes result for stable builds, but
>>>>> master is
>>>>> not always stable and sometimes we introduce new warnings in unstable
>>>>> builds(see builds 6310-6314).
>>>>> And findbugs and checkstyle will not fail unless we abort the building
>>>>> process I think, so it is safe to turn it on every time.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>>>>>
>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>>>>> >
>>>>> > Cheers
>>>>> >
>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com> wrote:
>>>>> >
>>>>> > > Going to change the config of HBase-TRUNK jenkins to show findbugs,
>>>>> > > checkstyle and jacoco coverage report.
>>>>> > > The config has been tested on
>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30
>>>>> times.
>>>>> > Not
>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
>>>>> overhead
>>>>> > of
>>>>> > > collecting information for code coverage).
>>>>> > > Thanks.
>>>>> > >
>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>>>>> > >
>>>>> > > > +1, thanks a lot for improving our build hygiene.
>>>>> > > >
>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <
>>>>> enis.soz@gmail.com>
>>>>> > > wrote:
>>>>> > > >
>>>>> > > > > Thanks Sean. This is great.
>>>>> > > > >
>>>>> > > > > Enis
>>>>> > > > >
>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <
>>>>> busbey@cloudera.com>
>>>>> > > > wrote:
>>>>> > > > >
>>>>> > > > > > FYI I just finished chasing down the breakage for mvn site
>>>>> on all
>>>>> > > patch
>>>>> > > > > > builds.
>>>>> > > > > >
>>>>> > > > > > HBASE-13191 consolidates the few places in the test-patch
>>>>> code
>>>>> > where
>>>>> > > we
>>>>> > > > > > hard coded MAVEN_OPTS.
>>>>> > > > > >
>>>>> > > > > > If you look at the PreCommit job now, we use the "set
>>>>> environment
>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything else
>>>>> > respects
>>>>> > > > > that
>>>>> > > > > > setting.
>>>>> > > > > >
>>>>> > > > > > I set the initial value to be a combination of the memory
>>>>> > limitations
>>>>> > > > > we've
>>>>> > > > > > been actually running with (the ~6G was getting ignored) and
>>>>> the
>>>>> > > > permgen
>>>>> > > > > > needed for site.
>>>>> > > > > >
>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
>>>>> > > > > >
>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net>
>>>>> wrote:
>>>>> > > > > >
>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
>>>>> patch-build
>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to
>>>>> be the
>>>>> > > same
>>>>> > > > as
>>>>> > > > > > > those of trunk build too, setting
>>>>> MAVEN_OPTS="-Xmx6100m"... it
>>>>> > had
>>>>> > > > been
>>>>> > > > > > > 3000.
>>>>> > > > > > >
>>>>> > > > > > > Yours,
>>>>> > > > > > > St.Ack
>>>>> > > > > > >
>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net>
>>>>> wrote:
>>>>> > > > > > >
>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds and
>>>>> or last
>>>>> > 7
>>>>> > > > > days,
>>>>> > > > > > > > whichever comes first.
>>>>> > > > > > > > St.Ack
>>>>> > > > > > > >
>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net>
>>>>> > wrote:
>>>>> > > > > > > >
>>>>> > > > > > > >> Branch-1 and master have stabilized and now run mostly
>>>>> blue
>>>>> > > (give
>>>>> > > > or
>>>>> > > > > > > take
>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1
>>>>> has
>>>>> > > helped
>>>>> > > > us
>>>>> > > > > > > >> identify at least one destabilizing commit in the last
>>>>> few
>>>>> > days,
>>>>> > > > > maybe
>>>>> > > > > > > two;
>>>>> > > > > > > >> this is as it should be (smile).
>>>>> > > > > > > >>
>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch, make
>>>>> sure
>>>>> > > > > subsequent
>>>>> > > > > > > >> builds stay blue. You can subscribe to
>>>>> > builds@hbase.apache.org
>>>>> > > to
>>>>> > > > > get
>>>>> > > > > > > >> notice of failures if not already subscribed.
>>>>> > > > > > > >>
>>>>> > > > > > > >> Thanks,
>>>>> > > > > > > >> St.Ack
>>>>> > > > > > > >>
>>>>> > > > > > > >> 1.
>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>>>>> > > > > > > >> 2.
>>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>>>>> > > > > > > >>
>>>>> > > > > > > >>
>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <
>>>>> stack@duboce.net>
>>>>> > > wrote:
>>>>> > > > > > > >>
>>>>> > > > > > > >>> A few notes on testing.
>>>>> > > > > > > >>>
>>>>> > > > > > > >>> Too long to read, infra is more capable now and after
>>>>> some
>>>>> > > work,
>>>>> > > > we
>>>>> > > > > > are
>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets
>>>>> try and
>>>>> > > keep
>>>>> > > > it
>>>>> > > > > > > this
>>>>> > > > > > > >>> way going forward.
>>>>> > > > > > > >>>
>>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
>>>>> > > > > > > >>>
>>>>> > > > > > > >>> A recent spurt of test fixing combined with more
>>>>> capable
>>>>> > > hardware
>>>>> > > > > > seems
>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
>>>>> passing
>>>>> > now
>>>>> > > on
>>>>> > > > > > > branch-1
>>>>> > > > > > > >>> and master. Lets try and keep it this way and start
>>>>> to trust
>>>>> > > our
>>>>> > > > > > test
>>>>> > > > > > > runs
>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and nail
>>>>> them.
>>>>> > > > > > > >>>
>>>>> > > > > > > >>> Our tests now run in parallel with other test suites
>>>>> where
>>>>> > > > previous
>>>>> > > > > > we
>>>>> > > > > > > >>> ran alone. You can see this sometimes when our zombie
>>>>> > detector
>>>>> > > > > > reports
>>>>> > > > > > > >>> tests from another project altogether as lingerers (To
>>>>> be
>>>>> > > fixed).
>>>>> > > > > > > Some of
>>>>> > > > > > > >>> our tests are failing because a concurrent hbase run is
>>>>> > undoing
>>>>> > > > > > > classes and
>>>>> > > > > > > >>> data from under it. Also, lets fix.
>>>>> > > > > > > >>>
>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them to
>>>>> > complete.
>>>>> > > > > Many
>>>>> > > > > > > >>> are heavy-duty integration tests starting up multiple
>>>>> > clusters
>>>>> > > > and
>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they
>>>>> pass at
>>>>> > all.
>>>>> > > > > > > Usually
>>>>> > > > > > > >>> integration tests have been cast as unit tests because
>>>>> there
>>>>> > > was
>>>>> > > > no
>>>>> > > > > > > where
>>>>> > > > > > > >>> else for them to get an airing. We have the hbase-it
>>>>> suite
>>>>> > now
>>>>> > > > > which
>>>>> > > > > > > would
>>>>> > > > > > > >>> be a more apt place but until these are run on a
>>>>> regular
>>>>> > basis
>>>>> > > in
>>>>> > > > > > > public
>>>>> > > > > > > >>> for all to see, the fat integration tests disguised as
>>>>> unit
>>>>> > > tests
>>>>> > > > > > will
>>>>> > > > > > > >>> remain. A review of our current unit tests weeding
>>>>> the old
>>>>> > > cruft
>>>>> > > > > and
>>>>> > > > > > > the
>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
>>>>> undertaking
>>>>> > if
>>>>> > > > > > > someone is
>>>>> > > > > > > >>> looking to contribute.
>>>>> > > > > > > >>>
>>>>> > > > > > > >>> Alex Newman has been working on making our tests work
>>>>> up on
>>>>> > > > travis
>>>>> > > > > > and
>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes end-to-end.
>>>>> He
>>>>> > also
>>>>> > > > > added
>>>>> > > > > > in
>>>>> > > > > > > >>> some "type" categorizations -- client, filter,
>>>>> mapreduce --
>>>>> > > > > alongside
>>>>> > > > > > > our
>>>>> > > > > > > >>> old "sizing" categorizations of small/medium/large.
>>>>> His
>>>>> > > thinking
>>>>> > > > > is
>>>>> > > > > > > that
>>>>> > > > > > > >>> we can run these categorizations in parallel so we
>>>>> could run
>>>>> > > the
>>>>> > > > > > total
>>>>> > > > > > > >>> suite in about the time of the longest test, say
>>>>> > 20-30minutes?
>>>>> > > > We
>>>>> > > > > > > could
>>>>> > > > > > > >>> even change Apache to run them this way.
>>>>> > > > > > > >>>
>>>>> > > > > > > >>> FYI,
>>>>> > > > > > > >>> St.Ack
>>>>> > > > > > > >>>
>>>>> > > > > > > >>>
>>>>> > > > > > > >>>
>>>>> > > > > > > >>>
>>>>> > > > > > > >>>
>>>>> > > > > > > >>>
>>>>> > > > > > > >>>
>>>>> > > > > > > >>
>>>>> > > > > > > >
>>>>> > > > > > >
>>>>> > > > > >
>>>>> > > > > >
>>>>> > > > > >
>>>>> > > > > > --
>>>>> > > > > > Sean
>>>>> > > > > >
>>>>> > > > >
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > > --
>>>>> > > > Best regards,
>>>>> > > >
>>>>> > > > - Andy
>>>>> > > >
>>>>> > > > Problems worthy of attack prove their worth by hitting back. -
>>>>> Piet
>>>>> > Hein
>>>>> > > > (via Tom White)
>>>>> > > >
>>>>> > >
>>>>> >
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sean
>>>>
>>>
>>>
>>>
>>> --
>>> Sean
>>>
>>
>>
>>
>> --
>> Sean
>>
>
>
>
> --
> Sean
>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
sigh. that should have been "is now a matrix build".
On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bu...@cloudera.com> wrote:
> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 in
> parallel.
>
> I saved the old job for now and left it disabled[2].
>
>
> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
> [2]:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>
> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> Sorry for the noise. I also updated the checkstyle step on HBase-TRUNK
>>
>> from
>> mvn checkstyle:checkstyle
>> to
>> mvn -DskipTests package checkstyle:checkstyle
>>
>> to deal with the same issue. Looks all clear now.
>>
>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com> wrote:
>>
>>> Updated the following builds:
>>>
>>> * HBase-TRUNK
>>>
>>> moved from
>>>
>>> mvn clean compile findbugs:findbugs
>>>
>>> to
>>>
>>> mvn clean -DskipTests package findbugs:findbugs
>>>
>>> To work around the known issue where we can't do a bootstrap compile
>>> without previous install or remote SNAPSHOT artifacts. (recently triggered
>>> by the addition of the procedure module on master)
>>>
>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
>>>
>>>> Make findbugs and checkstyle plugins always run.
>>>> The default behavior only publishes result for stable builds, but
>>>> master is
>>>> not always stable and sometimes we introduce new warnings in unstable
>>>> builds(see builds 6310-6314).
>>>> And findbugs and checkstyle will not fail unless we abort the building
>>>> process I think, so it is safe to turn it on every time.
>>>>
>>>> Thanks.
>>>>
>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>>>>
>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>>>> >
>>>> > Cheers
>>>> >
>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com> wrote:
>>>> >
>>>> > > Going to change the config of HBase-TRUNK jenkins to show findbugs,
>>>> > > checkstyle and jacoco coverage report.
>>>> > > The config has been tested on
>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30
>>>> times.
>>>> > Not
>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
>>>> overhead
>>>> > of
>>>> > > collecting information for code coverage).
>>>> > > Thanks.
>>>> > >
>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>>>> > >
>>>> > > > +1, thanks a lot for improving our build hygiene.
>>>> > > >
>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <
>>>> enis.soz@gmail.com>
>>>> > > wrote:
>>>> > > >
>>>> > > > > Thanks Sean. This is great.
>>>> > > > >
>>>> > > > > Enis
>>>> > > > >
>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <
>>>> busbey@cloudera.com>
>>>> > > > wrote:
>>>> > > > >
>>>> > > > > > FYI I just finished chasing down the breakage for mvn site on
>>>> all
>>>> > > patch
>>>> > > > > > builds.
>>>> > > > > >
>>>> > > > > > HBASE-13191 consolidates the few places in the test-patch code
>>>> > where
>>>> > > we
>>>> > > > > > hard coded MAVEN_OPTS.
>>>> > > > > >
>>>> > > > > > If you look at the PreCommit job now, we use the "set
>>>> environment
>>>> > > > > > variables" option to set MAVEN_OPTS and then everything else
>>>> > respects
>>>> > > > > that
>>>> > > > > > setting.
>>>> > > > > >
>>>> > > > > > I set the initial value to be a combination of the memory
>>>> > limitations
>>>> > > > > we've
>>>> > > > > > been actually running with (the ~6G was getting ignored) and
>>>> the
>>>> > > > permgen
>>>> > > > > > needed for site.
>>>> > > > > >
>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
>>>> > > > > >
>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net>
>>>> wrote:
>>>> > > > > >
>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
>>>> patch-build
>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be
>>>> the
>>>> > > same
>>>> > > > as
>>>> > > > > > > those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"...
>>>> it
>>>> > had
>>>> > > > been
>>>> > > > > > > 3000.
>>>> > > > > > >
>>>> > > > > > > Yours,
>>>> > > > > > > St.Ack
>>>> > > > > > >
>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net>
>>>> wrote:
>>>> > > > > > >
>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds and or
>>>> last
>>>> > 7
>>>> > > > > days,
>>>> > > > > > > > whichever comes first.
>>>> > > > > > > > St.Ack
>>>> > > > > > > >
>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net>
>>>> > wrote:
>>>> > > > > > > >
>>>> > > > > > > >> Branch-1 and master have stabilized and now run mostly
>>>> blue
>>>> > > (give
>>>> > > > or
>>>> > > > > > > take
>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1
>>>> has
>>>> > > helped
>>>> > > > us
>>>> > > > > > > >> identify at least one destabilizing commit in the last
>>>> few
>>>> > days,
>>>> > > > > maybe
>>>> > > > > > > two;
>>>> > > > > > > >> this is as it should be (smile).
>>>> > > > > > > >>
>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch, make
>>>> sure
>>>> > > > > subsequent
>>>> > > > > > > >> builds stay blue. You can subscribe to
>>>> > builds@hbase.apache.org
>>>> > > to
>>>> > > > > get
>>>> > > > > > > >> notice of failures if not already subscribed.
>>>> > > > > > > >>
>>>> > > > > > > >> Thanks,
>>>> > > > > > > >> St.Ack
>>>> > > > > > > >>
>>>> > > > > > > >> 1.
>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>>>> > > > > > > >> 2.
>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>>>> > > > > > > >>
>>>> > > > > > > >>
>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <stack@duboce.net
>>>> >
>>>> > > wrote:
>>>> > > > > > > >>
>>>> > > > > > > >>> A few notes on testing.
>>>> > > > > > > >>>
>>>> > > > > > > >>> Too long to read, infra is more capable now and after
>>>> some
>>>> > > work,
>>>> > > > we
>>>> > > > > > are
>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets try
>>>> and
>>>> > > keep
>>>> > > > it
>>>> > > > > > > this
>>>> > > > > > > >>> way going forward.
>>>> > > > > > > >>>
>>>> > > > > > > >>> Apache Infra has new, more capable hardware.
>>>> > > > > > > >>>
>>>> > > > > > > >>> A recent spurt of test fixing combined with more capable
>>>> > > hardware
>>>> > > > > > seems
>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
>>>> passing
>>>> > now
>>>> > > on
>>>> > > > > > > branch-1
>>>> > > > > > > >>> and master. Lets try and keep it this way and start to
>>>> trust
>>>> > > our
>>>> > > > > > test
>>>> > > > > > > runs
>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and nail
>>>> them.
>>>> > > > > > > >>>
>>>> > > > > > > >>> Our tests now run in parallel with other test suites
>>>> where
>>>> > > > previous
>>>> > > > > > we
>>>> > > > > > > >>> ran alone. You can see this sometimes when our zombie
>>>> > detector
>>>> > > > > > reports
>>>> > > > > > > >>> tests from another project altogether as lingerers (To
>>>> be
>>>> > > fixed).
>>>> > > > > > > Some of
>>>> > > > > > > >>> our tests are failing because a concurrent hbase run is
>>>> > undoing
>>>> > > > > > > classes and
>>>> > > > > > > >>> data from under it. Also, lets fix.
>>>> > > > > > > >>>
>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them to
>>>> > complete.
>>>> > > > > Many
>>>> > > > > > > >>> are heavy-duty integration tests starting up multiple
>>>> > clusters
>>>> > > > and
>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they pass
>>>> at
>>>> > all.
>>>> > > > > > > Usually
>>>> > > > > > > >>> integration tests have been cast as unit tests because
>>>> there
>>>> > > was
>>>> > > > no
>>>> > > > > > > where
>>>> > > > > > > >>> else for them to get an airing. We have the hbase-it
>>>> suite
>>>> > now
>>>> > > > > which
>>>> > > > > > > would
>>>> > > > > > > >>> be a more apt place but until these are run on a regular
>>>> > basis
>>>> > > in
>>>> > > > > > > public
>>>> > > > > > > >>> for all to see, the fat integration tests disguised as
>>>> unit
>>>> > > tests
>>>> > > > > > will
>>>> > > > > > > >>> remain. A review of our current unit tests weeding the
>>>> old
>>>> > > cruft
>>>> > > > > and
>>>> > > > > > > the
>>>> > > > > > > >>> no longer relevant or duplicates would be a nice
>>>> undertaking
>>>> > if
>>>> > > > > > > someone is
>>>> > > > > > > >>> looking to contribute.
>>>> > > > > > > >>>
>>>> > > > > > > >>> Alex Newman has been working on making our tests work
>>>> up on
>>>> > > > travis
>>>> > > > > > and
>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes end-to-end.
>>>> He
>>>> > also
>>>> > > > > added
>>>> > > > > > in
>>>> > > > > > > >>> some "type" categorizations -- client, filter,
>>>> mapreduce --
>>>> > > > > alongside
>>>> > > > > > > our
>>>> > > > > > > >>> old "sizing" categorizations of small/medium/large. His
>>>> > > thinking
>>>> > > > > is
>>>> > > > > > > that
>>>> > > > > > > >>> we can run these categorizations in parallel so we
>>>> could run
>>>> > > the
>>>> > > > > > total
>>>> > > > > > > >>> suite in about the time of the longest test, say
>>>> > 20-30minutes?
>>>> > > > We
>>>> > > > > > > could
>>>> > > > > > > >>> even change Apache to run them this way.
>>>> > > > > > > >>>
>>>> > > > > > > >>> FYI,
>>>> > > > > > > >>> St.Ack
>>>> > > > > > > >>>
>>>> > > > > > > >>>
>>>> > > > > > > >>>
>>>> > > > > > > >>>
>>>> > > > > > > >>>
>>>> > > > > > > >>>
>>>> > > > > > > >>>
>>>> > > > > > > >>
>>>> > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > > >
>>>> > > > > >
>>>> > > > > > --
>>>> > > > > > Sean
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > > > --
>>>> > > > Best regards,
>>>> > > >
>>>> > > > - Andy
>>>> > > >
>>>> > > > Problems worthy of attack prove their worth by hitting back. -
>>>> Piet
>>>> > Hein
>>>> > > > (via Tom White)
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>
>>>
>>>
>>> --
>>> Sean
>>>
>>
>>
>>
>> --
>> Sean
>>
>
>
>
> --
> Sean
>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 in
parallel.
I saved the old job for now and left it disabled[2].
[1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
[2]:
https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bu...@cloudera.com> wrote:
> Sorry for the noise. I also updated the checkstyle step on HBase-TRUNK
>
> from
> mvn checkstyle:checkstyle
> to
> mvn -DskipTests package checkstyle:checkstyle
>
> to deal with the same issue. Looks all clear now.
>
> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
>> Updated the following builds:
>>
>> * HBase-TRUNK
>>
>> moved from
>>
>> mvn clean compile findbugs:findbugs
>>
>> to
>>
>> mvn clean -DskipTests package findbugs:findbugs
>>
>> To work around the known issue where we can't do a bootstrap compile
>> without previous install or remote SNAPSHOT artifacts. (recently triggered
>> by the addition of the procedure module on master)
>>
>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
>>
>>> Make findbugs and checkstyle plugins always run.
>>> The default behavior only publishes result for stable builds, but master
>>> is
>>> not always stable and sometimes we introduce new warnings in unstable
>>> builds(see builds 6310-6314).
>>> And findbugs and checkstyle will not fail unless we abort the building
>>> process I think, so it is safe to turn it on every time.
>>>
>>> Thanks.
>>>
>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>>>
>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>>> >
>>> > Cheers
>>> >
>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com> wrote:
>>> >
>>> > > Going to change the config of HBase-TRUNK jenkins to show findbugs,
>>> > > checkstyle and jacoco coverage report.
>>> > > The config has been tested on
>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30
>>> times.
>>> > Not
>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
>>> overhead
>>> > of
>>> > > collecting information for code coverage).
>>> > > Thanks.
>>> > >
>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>>> > >
>>> > > > +1, thanks a lot for improving our build hygiene.
>>> > > >
>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <enis.soz@gmail.com
>>> >
>>> > > wrote:
>>> > > >
>>> > > > > Thanks Sean. This is great.
>>> > > > >
>>> > > > > Enis
>>> > > > >
>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <
>>> busbey@cloudera.com>
>>> > > > wrote:
>>> > > > >
>>> > > > > > FYI I just finished chasing down the breakage for mvn site on
>>> all
>>> > > patch
>>> > > > > > builds.
>>> > > > > >
>>> > > > > > HBASE-13191 consolidates the few places in the test-patch code
>>> > where
>>> > > we
>>> > > > > > hard coded MAVEN_OPTS.
>>> > > > > >
>>> > > > > > If you look at the PreCommit job now, we use the "set
>>> environment
>>> > > > > > variables" option to set MAVEN_OPTS and then everything else
>>> > respects
>>> > > > > that
>>> > > > > > setting.
>>> > > > > >
>>> > > > > > I set the initial value to be a combination of the memory
>>> > limitations
>>> > > > > we've
>>> > > > > > been actually running with (the ~6G was getting ignored) and
>>> the
>>> > > > permgen
>>> > > > > > needed for site.
>>> > > > > >
>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
>>> > > > > >
>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net>
>>> wrote:
>>> > > > > >
>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
>>> patch-build
>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be
>>> the
>>> > > same
>>> > > > as
>>> > > > > > > those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"...
>>> it
>>> > had
>>> > > > been
>>> > > > > > > 3000.
>>> > > > > > >
>>> > > > > > > Yours,
>>> > > > > > > St.Ack
>>> > > > > > >
>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net>
>>> wrote:
>>> > > > > > >
>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds and or
>>> last
>>> > 7
>>> > > > > days,
>>> > > > > > > > whichever comes first.
>>> > > > > > > > St.Ack
>>> > > > > > > >
>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net>
>>> > wrote:
>>> > > > > > > >
>>> > > > > > > >> Branch-1 and master have stabilized and now run mostly
>>> blue
>>> > > (give
>>> > > > or
>>> > > > > > > take
>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1 has
>>> > > helped
>>> > > > us
>>> > > > > > > >> identify at least one destabilizing commit in the last few
>>> > days,
>>> > > > > maybe
>>> > > > > > > two;
>>> > > > > > > >> this is as it should be (smile).
>>> > > > > > > >>
>>> > > > > > > >> Lets keep our builds blue. If you commit a patch, make
>>> sure
>>> > > > > subsequent
>>> > > > > > > >> builds stay blue. You can subscribe to
>>> > builds@hbase.apache.org
>>> > > to
>>> > > > > get
>>> > > > > > > >> notice of failures if not already subscribed.
>>> > > > > > > >>
>>> > > > > > > >> Thanks,
>>> > > > > > > >> St.Ack
>>> > > > > > > >>
>>> > > > > > > >> 1.
>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>>> > > > > > > >> 2.
>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <st...@duboce.net>
>>> > > wrote:
>>> > > > > > > >>
>>> > > > > > > >>> A few notes on testing.
>>> > > > > > > >>>
>>> > > > > > > >>> Too long to read, infra is more capable now and after
>>> some
>>> > > work,
>>> > > > we
>>> > > > > > are
>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets try
>>> and
>>> > > keep
>>> > > > it
>>> > > > > > > this
>>> > > > > > > >>> way going forward.
>>> > > > > > > >>>
>>> > > > > > > >>> Apache Infra has new, more capable hardware.
>>> > > > > > > >>>
>>> > > > > > > >>> A recent spurt of test fixing combined with more capable
>>> > > hardware
>>> > > > > > seems
>>> > > > > > > >>> to have gotten us to a new place; tests are mostly
>>> passing
>>> > now
>>> > > on
>>> > > > > > > branch-1
>>> > > > > > > >>> and master. Lets try and keep it this way and start to
>>> trust
>>> > > our
>>> > > > > > test
>>> > > > > > > runs
>>> > > > > > > >>> again. Just a few flakies remain. Lets try and nail
>>> them.
>>> > > > > > > >>>
>>> > > > > > > >>> Our tests now run in parallel with other test suites
>>> where
>>> > > > previous
>>> > > > > > we
>>> > > > > > > >>> ran alone. You can see this sometimes when our zombie
>>> > detector
>>> > > > > > reports
>>> > > > > > > >>> tests from another project altogether as lingerers (To be
>>> > > fixed).
>>> > > > > > > Some of
>>> > > > > > > >>> our tests are failing because a concurrent hbase run is
>>> > undoing
>>> > > > > > > classes and
>>> > > > > > > >>> data from under it. Also, lets fix.
>>> > > > > > > >>>
>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them to
>>> > complete.
>>> > > > > Many
>>> > > > > > > >>> are heavy-duty integration tests starting up multiple
>>> > clusters
>>> > > > and
>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they pass
>>> at
>>> > all.
>>> > > > > > > Usually
>>> > > > > > > >>> integration tests have been cast as unit tests because
>>> there
>>> > > was
>>> > > > no
>>> > > > > > > where
>>> > > > > > > >>> else for them to get an airing. We have the hbase-it
>>> suite
>>> > now
>>> > > > > which
>>> > > > > > > would
>>> > > > > > > >>> be a more apt place but until these are run on a regular
>>> > basis
>>> > > in
>>> > > > > > > public
>>> > > > > > > >>> for all to see, the fat integration tests disguised as
>>> unit
>>> > > tests
>>> > > > > > will
>>> > > > > > > >>> remain. A review of our current unit tests weeding the
>>> old
>>> > > cruft
>>> > > > > and
>>> > > > > > > the
>>> > > > > > > >>> no longer relevant or duplicates would be a nice
>>> undertaking
>>> > if
>>> > > > > > > someone is
>>> > > > > > > >>> looking to contribute.
>>> > > > > > > >>>
>>> > > > > > > >>> Alex Newman has been working on making our tests work up
>>> on
>>> > > > travis
>>> > > > > > and
>>> > > > > > > >>> circle-ci. That'll be sweet when it goes end-to-end. He
>>> > also
>>> > > > > added
>>> > > > > > in
>>> > > > > > > >>> some "type" categorizations -- client, filter, mapreduce
>>> --
>>> > > > > alongside
>>> > > > > > > our
>>> > > > > > > >>> old "sizing" categorizations of small/medium/large. His
>>> > > thinking
>>> > > > > is
>>> > > > > > > that
>>> > > > > > > >>> we can run these categorizations in parallel so we could
>>> run
>>> > > the
>>> > > > > > total
>>> > > > > > > >>> suite in about the time of the longest test, say
>>> > 20-30minutes?
>>> > > > We
>>> > > > > > > could
>>> > > > > > > >>> even change Apache to run them this way.
>>> > > > > > > >>>
>>> > > > > > > >>> FYI,
>>> > > > > > > >>> St.Ack
>>> > > > > > > >>>
>>> > > > > > > >>>
>>> > > > > > > >>>
>>> > > > > > > >>>
>>> > > > > > > >>>
>>> > > > > > > >>>
>>> > > > > > > >>>
>>> > > > > > > >>
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > --
>>> > > > > > Sean
>>> > > > > >
>>> > > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > > --
>>> > > > Best regards,
>>> > > >
>>> > > > - Andy
>>> > > >
>>> > > > Problems worthy of attack prove their worth by hitting back. - Piet
>>> > Hein
>>> > > > (via Tom White)
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>>
>> --
>> Sean
>>
>
>
>
> --
> Sean
>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
Sorry for the noise. I also updated the checkstyle step on HBase-TRUNK
from
mvn checkstyle:checkstyle
to
mvn -DskipTests package checkstyle:checkstyle
to deal with the same issue. Looks all clear now.
On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bu...@cloudera.com> wrote:
> Updated the following builds:
>
> * HBase-TRUNK
>
> moved from
>
> mvn clean compile findbugs:findbugs
>
> to
>
> mvn clean -DskipTests package findbugs:findbugs
>
> To work around the known issue where we can't do a bootstrap compile
> without previous install or remote SNAPSHOT artifacts. (recently triggered
> by the addition of the procedure module on master)
>
> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
>
>> Make findbugs and checkstyle plugins always run.
>> The default behavior only publishes result for stable builds, but master
>> is
>> not always stable and sometimes we introduce new warnings in unstable
>> builds(see builds 6310-6314).
>> And findbugs and checkstyle will not fail unless we abort the building
>> process I think, so it is safe to turn it on every time.
>>
>> Thanks.
>>
>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>>
>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>> >
>> > Cheers
>> >
>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com> wrote:
>> >
>> > > Going to change the config of HBase-TRUNK jenkins to show findbugs,
>> > > checkstyle and jacoco coverage report.
>> > > The config has been tested on
>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30 times.
>> > Not
>> > > much different from HBase-TRUNK unless it runs ~30% slower(the
>> overhead
>> > of
>> > > collecting information for code coverage).
>> > > Thanks.
>> > >
>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>> > >
>> > > > +1, thanks a lot for improving our build hygiene.
>> > > >
>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <en...@gmail.com>
>> > > wrote:
>> > > >
>> > > > > Thanks Sean. This is great.
>> > > > >
>> > > > > Enis
>> > > > >
>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <busbey@cloudera.com
>> >
>> > > > wrote:
>> > > > >
>> > > > > > FYI I just finished chasing down the breakage for mvn site on
>> all
>> > > patch
>> > > > > > builds.
>> > > > > >
>> > > > > > HBASE-13191 consolidates the few places in the test-patch code
>> > where
>> > > we
>> > > > > > hard coded MAVEN_OPTS.
>> > > > > >
>> > > > > > If you look at the PreCommit job now, we use the "set
>> environment
>> > > > > > variables" option to set MAVEN_OPTS and then everything else
>> > respects
>> > > > > that
>> > > > > > setting.
>> > > > > >
>> > > > > > I set the initial value to be a combination of the memory
>> > limitations
>> > > > > we've
>> > > > > > been actually running with (the ~6G was getting ignored) and the
>> > > > permgen
>> > > > > > needed for site.
>> > > > > >
>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
>> > > > > >
>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net>
>> wrote:
>> > > > > >
>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
>> patch-build
>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be
>> the
>> > > same
>> > > > as
>> > > > > > > those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"... it
>> > had
>> > > > been
>> > > > > > > 3000.
>> > > > > > >
>> > > > > > > Yours,
>> > > > > > > St.Ack
>> > > > > > >
>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net>
>> wrote:
>> > > > > > >
>> > > > > > > > I upped hadoopqa retention to keep last 100 builds and or
>> last
>> > 7
>> > > > > days,
>> > > > > > > > whichever comes first.
>> > > > > > > > St.Ack
>> > > > > > > >
>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net>
>> > wrote:
>> > > > > > > >
>> > > > > > > >> Branch-1 and master have stabilized and now run mostly blue
>> > > (give
>> > > > or
>> > > > > > > take
>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1 has
>> > > helped
>> > > > us
>> > > > > > > >> identify at least one destabilizing commit in the last few
>> > days,
>> > > > > maybe
>> > > > > > > two;
>> > > > > > > >> this is as it should be (smile).
>> > > > > > > >>
>> > > > > > > >> Lets keep our builds blue. If you commit a patch, make sure
>> > > > > subsequent
>> > > > > > > >> builds stay blue. You can subscribe to
>> > builds@hbase.apache.org
>> > > to
>> > > > > get
>> > > > > > > >> notice of failures if not already subscribed.
>> > > > > > > >>
>> > > > > > > >> Thanks,
>> > > > > > > >> St.Ack
>> > > > > > > >>
>> > > > > > > >> 1.
>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>> > > > > > > >> 2.
>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <st...@duboce.net>
>> > > wrote:
>> > > > > > > >>
>> > > > > > > >>> A few notes on testing.
>> > > > > > > >>>
>> > > > > > > >>> Too long to read, infra is more capable now and after some
>> > > work,
>> > > > we
>> > > > > > are
>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets try
>> and
>> > > keep
>> > > > it
>> > > > > > > this
>> > > > > > > >>> way going forward.
>> > > > > > > >>>
>> > > > > > > >>> Apache Infra has new, more capable hardware.
>> > > > > > > >>>
>> > > > > > > >>> A recent spurt of test fixing combined with more capable
>> > > hardware
>> > > > > > seems
>> > > > > > > >>> to have gotten us to a new place; tests are mostly passing
>> > now
>> > > on
>> > > > > > > branch-1
>> > > > > > > >>> and master. Lets try and keep it this way and start to
>> trust
>> > > our
>> > > > > > test
>> > > > > > > runs
>> > > > > > > >>> again. Just a few flakies remain. Lets try and nail
>> them.
>> > > > > > > >>>
>> > > > > > > >>> Our tests now run in parallel with other test suites where
>> > > > previous
>> > > > > > we
>> > > > > > > >>> ran alone. You can see this sometimes when our zombie
>> > detector
>> > > > > > reports
>> > > > > > > >>> tests from another project altogether as lingerers (To be
>> > > fixed).
>> > > > > > > Some of
>> > > > > > > >>> our tests are failing because a concurrent hbase run is
>> > undoing
>> > > > > > > classes and
>> > > > > > > >>> data from under it. Also, lets fix.
>> > > > > > > >>>
>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them to
>> > complete.
>> > > > > Many
>> > > > > > > >>> are heavy-duty integration tests starting up multiple
>> > clusters
>> > > > and
>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they pass at
>> > all.
>> > > > > > > Usually
>> > > > > > > >>> integration tests have been cast as unit tests because
>> there
>> > > was
>> > > > no
>> > > > > > > where
>> > > > > > > >>> else for them to get an airing. We have the hbase-it
>> suite
>> > now
>> > > > > which
>> > > > > > > would
>> > > > > > > >>> be a more apt place but until these are run on a regular
>> > basis
>> > > in
>> > > > > > > public
>> > > > > > > >>> for all to see, the fat integration tests disguised as
>> unit
>> > > tests
>> > > > > > will
>> > > > > > > >>> remain. A review of our current unit tests weeding the
>> old
>> > > cruft
>> > > > > and
>> > > > > > > the
>> > > > > > > >>> no longer relevant or duplicates would be a nice
>> undertaking
>> > if
>> > > > > > > someone is
>> > > > > > > >>> looking to contribute.
>> > > > > > > >>>
>> > > > > > > >>> Alex Newman has been working on making our tests work up
>> on
>> > > > travis
>> > > > > > and
>> > > > > > > >>> circle-ci. That'll be sweet when it goes end-to-end. He
>> > also
>> > > > > added
>> > > > > > in
>> > > > > > > >>> some "type" categorizations -- client, filter, mapreduce
>> --
>> > > > > alongside
>> > > > > > > our
>> > > > > > > >>> old "sizing" categorizations of small/medium/large. His
>> > > thinking
>> > > > > is
>> > > > > > > that
>> > > > > > > >>> we can run these categorizations in parallel so we could
>> run
>> > > the
>> > > > > > total
>> > > > > > > >>> suite in about the time of the longest test, say
>> > 20-30minutes?
>> > > > We
>> > > > > > > could
>> > > > > > > >>> even change Apache to run them this way.
>> > > > > > > >>>
>> > > > > > > >>> FYI,
>> > > > > > > >>> St.Ack
>> > > > > > > >>>
>> > > > > > > >>>
>> > > > > > > >>>
>> > > > > > > >>>
>> > > > > > > >>>
>> > > > > > > >>>
>> > > > > > > >>>
>> > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > Sean
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Best regards,
>> > > >
>> > > > - Andy
>> > > >
>> > > > Problems worthy of attack prove their worth by hitting back. - Piet
>> > Hein
>> > > > (via Tom White)
>> > > >
>> > >
>> >
>>
>
>
>
> --
> Sean
>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
Updated the following builds:
* HBase-TRUNK
moved from
mvn clean compile findbugs:findbugs
to
mvn clean -DskipTests package findbugs:findbugs
To work around the known issue where we can't do a bootstrap compile
without previous install or remote SNAPSHOT artifacts. (recently triggered
by the addition of the procedure module on master)
On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <pa...@gmail.com> wrote:
> Make findbugs and checkstyle plugins always run.
> The default behavior only publishes result for stable builds, but master is
> not always stable and sometimes we introduce new warnings in unstable
> builds(see builds 6310-6314).
> And findbugs and checkstyle will not fail unless we abort the building
> process I think, so it is safe to turn it on every time.
>
> Thanks.
>
> 2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
>
> > +1 on letting HBase-TRUNK jenkins show coverage report.
> >
> > Cheers
> >
> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com> wrote:
> >
> > > Going to change the config of HBase-TRUNK jenkins to show findbugs,
> > > checkstyle and jacoco coverage report.
> > > The config has been tested on
> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30 times.
> > Not
> > > much different from HBase-TRUNK unless it runs ~30% slower(the overhead
> > of
> > > collecting information for code coverage).
> > > Thanks.
> > >
> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
> > >
> > > > +1, thanks a lot for improving our build hygiene.
> > > >
> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <en...@gmail.com>
> > > wrote:
> > > >
> > > > > Thanks Sean. This is great.
> > > > >
> > > > > Enis
> > > > >
> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <bu...@cloudera.com>
> > > > wrote:
> > > > >
> > > > > > FYI I just finished chasing down the breakage for mvn site on all
> > > patch
> > > > > > builds.
> > > > > >
> > > > > > HBASE-13191 consolidates the few places in the test-patch code
> > where
> > > we
> > > > > > hard coded MAVEN_OPTS.
> > > > > >
> > > > > > If you look at the PreCommit job now, we use the "set environment
> > > > > > variables" option to set MAVEN_OPTS and then everything else
> > respects
> > > > > that
> > > > > > setting.
> > > > > >
> > > > > > I set the initial value to be a combination of the memory
> > limitations
> > > > > we've
> > > > > > been actually running with (the ~6G was getting ignored) and the
> > > > permgen
> > > > > > needed for site.
> > > > > >
> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
> > > > > >
> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net>
> wrote:
> > > > > >
> > > > > > > I just made TRUNK and branch-1 builds use same jvm as
> patch-build
> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be the
> > > same
> > > > as
> > > > > > > those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"... it
> > had
> > > > been
> > > > > > > 3000.
> > > > > > >
> > > > > > > Yours,
> > > > > > > St.Ack
> > > > > > >
> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net>
> wrote:
> > > > > > >
> > > > > > > > I upped hadoopqa retention to keep last 100 builds and or
> last
> > 7
> > > > > days,
> > > > > > > > whichever comes first.
> > > > > > > > St.Ack
> > > > > > > >
> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net>
> > wrote:
> > > > > > > >
> > > > > > > >> Branch-1 and master have stabilized and now run mostly blue
> > > (give
> > > > or
> > > > > > > take
> > > > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1 has
> > > helped
> > > > us
> > > > > > > >> identify at least one destabilizing commit in the last few
> > days,
> > > > > maybe
> > > > > > > two;
> > > > > > > >> this is as it should be (smile).
> > > > > > > >>
> > > > > > > >> Lets keep our builds blue. If you commit a patch, make sure
> > > > > subsequent
> > > > > > > >> builds stay blue. You can subscribe to
> > builds@hbase.apache.org
> > > to
> > > > > get
> > > > > > > >> notice of failures if not already subscribed.
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> St.Ack
> > > > > > > >>
> > > > > > > >> 1.
> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> > > > > > > >> 2.
> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <st...@duboce.net>
> > > wrote:
> > > > > > > >>
> > > > > > > >>> A few notes on testing.
> > > > > > > >>>
> > > > > > > >>> Too long to read, infra is more capable now and after some
> > > work,
> > > > we
> > > > > > are
> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets try and
> > > keep
> > > > it
> > > > > > > this
> > > > > > > >>> way going forward.
> > > > > > > >>>
> > > > > > > >>> Apache Infra has new, more capable hardware.
> > > > > > > >>>
> > > > > > > >>> A recent spurt of test fixing combined with more capable
> > > hardware
> > > > > > seems
> > > > > > > >>> to have gotten us to a new place; tests are mostly passing
> > now
> > > on
> > > > > > > branch-1
> > > > > > > >>> and master. Lets try and keep it this way and start to
> trust
> > > our
> > > > > > test
> > > > > > > runs
> > > > > > > >>> again. Just a few flakies remain. Lets try and nail them.
> > > > > > > >>>
> > > > > > > >>> Our tests now run in parallel with other test suites where
> > > > previous
> > > > > > we
> > > > > > > >>> ran alone. You can see this sometimes when our zombie
> > detector
> > > > > > reports
> > > > > > > >>> tests from another project altogether as lingerers (To be
> > > fixed).
> > > > > > > Some of
> > > > > > > >>> our tests are failing because a concurrent hbase run is
> > undoing
> > > > > > > classes and
> > > > > > > >>> data from under it. Also, lets fix.
> > > > > > > >>>
> > > > > > > >>> Our tests are brittle. It takes 75minutes for them to
> > complete.
> > > > > Many
> > > > > > > >>> are heavy-duty integration tests starting up multiple
> > clusters
> > > > and
> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they pass at
> > all.
> > > > > > > Usually
> > > > > > > >>> integration tests have been cast as unit tests because
> there
> > > was
> > > > no
> > > > > > > where
> > > > > > > >>> else for them to get an airing. We have the hbase-it suite
> > now
> > > > > which
> > > > > > > would
> > > > > > > >>> be a more apt place but until these are run on a regular
> > basis
> > > in
> > > > > > > public
> > > > > > > >>> for all to see, the fat integration tests disguised as unit
> > > tests
> > > > > > will
> > > > > > > >>> remain. A review of our current unit tests weeding the old
> > > cruft
> > > > > and
> > > > > > > the
> > > > > > > >>> no longer relevant or duplicates would be a nice
> undertaking
> > if
> > > > > > > someone is
> > > > > > > >>> looking to contribute.
> > > > > > > >>>
> > > > > > > >>> Alex Newman has been working on making our tests work up on
> > > > travis
> > > > > > and
> > > > > > > >>> circle-ci. That'll be sweet when it goes end-to-end. He
> > also
> > > > > added
> > > > > > in
> > > > > > > >>> some "type" categorizations -- client, filter, mapreduce --
> > > > > alongside
> > > > > > > our
> > > > > > > >>> old "sizing" categorizations of small/medium/large. His
> > > thinking
> > > > > is
> > > > > > > that
> > > > > > > >>> we can run these categorizations in parallel so we could
> run
> > > the
> > > > > > total
> > > > > > > >>> suite in about the time of the longest test, say
> > 20-30minutes?
> > > > We
> > > > > > > could
> > > > > > > >>> even change Apache to run them this way.
> > > > > > > >>>
> > > > > > > >>> FYI,
> > > > > > > >>> St.Ack
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Sean
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > > - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > >
> >
>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by 张铎 <pa...@gmail.com>.
Make findbugs and checkstyle plugins always run.
The default behavior only publishes result for stable builds, but master is
not always stable and sometimes we introduce new warnings in unstable
builds(see builds 6310-6314).
And findbugs and checkstyle will not fail unless we abort the building
process I think, so it is safe to turn it on every time.
Thanks.
2015-03-22 22:44 GMT+08:00 Ted Yu <yu...@gmail.com>:
> +1 on letting HBase-TRUNK jenkins show coverage report.
>
> Cheers
>
> On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com> wrote:
>
> > Going to change the config of HBase-TRUNK jenkins to show findbugs,
> > checkstyle and jacoco coverage report.
> > The config has been tested on
> > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30 times.
> Not
> > much different from HBase-TRUNK unless it runs ~30% slower(the overhead
> of
> > collecting information for code coverage).
> > Thanks.
> >
> > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
> >
> > > +1, thanks a lot for improving our build hygiene.
> > >
> > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <en...@gmail.com>
> > wrote:
> > >
> > > > Thanks Sean. This is great.
> > > >
> > > > Enis
> > > >
> > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <bu...@cloudera.com>
> > > wrote:
> > > >
> > > > > FYI I just finished chasing down the breakage for mvn site on all
> > patch
> > > > > builds.
> > > > >
> > > > > HBASE-13191 consolidates the few places in the test-patch code
> where
> > we
> > > > > hard coded MAVEN_OPTS.
> > > > >
> > > > > If you look at the PreCommit job now, we use the "set environment
> > > > > variables" option to set MAVEN_OPTS and then everything else
> respects
> > > > that
> > > > > setting.
> > > > >
> > > > > I set the initial value to be a combination of the memory
> limitations
> > > > we've
> > > > > been actually running with (the ~6G was getting ignored) and the
> > > permgen
> > > > > needed for site.
> > > > >
> > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
> > > > >
> > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net> wrote:
> > > > >
> > > > > > I just made TRUNK and branch-1 builds use same jvm as patch-build
> > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be the
> > same
> > > as
> > > > > > those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"... it
> had
> > > been
> > > > > > 3000.
> > > > > >
> > > > > > Yours,
> > > > > > St.Ack
> > > > > >
> > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net> wrote:
> > > > > >
> > > > > > > I upped hadoopqa retention to keep last 100 builds and or last
> 7
> > > > days,
> > > > > > > whichever comes first.
> > > > > > > St.Ack
> > > > > > >
> > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net>
> wrote:
> > > > > > >
> > > > > > >> Branch-1 and master have stabilized and now run mostly blue
> > (give
> > > or
> > > > > > take
> > > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1 has
> > helped
> > > us
> > > > > > >> identify at least one destabilizing commit in the last few
> days,
> > > > maybe
> > > > > > two;
> > > > > > >> this is as it should be (smile).
> > > > > > >>
> > > > > > >> Lets keep our builds blue. If you commit a patch, make sure
> > > > subsequent
> > > > > > >> builds stay blue. You can subscribe to
> builds@hbase.apache.org
> > to
> > > > get
> > > > > > >> notice of failures if not already subscribed.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> St.Ack
> > > > > > >>
> > > > > > >> 1.
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> > > > > > >> 2.
> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> > > > > > >>
> > > > > > >>
> > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <st...@duboce.net>
> > wrote:
> > > > > > >>
> > > > > > >>> A few notes on testing.
> > > > > > >>>
> > > > > > >>> Too long to read, infra is more capable now and after some
> > work,
> > > we
> > > > > are
> > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets try and
> > keep
> > > it
> > > > > > this
> > > > > > >>> way going forward.
> > > > > > >>>
> > > > > > >>> Apache Infra has new, more capable hardware.
> > > > > > >>>
> > > > > > >>> A recent spurt of test fixing combined with more capable
> > hardware
> > > > > seems
> > > > > > >>> to have gotten us to a new place; tests are mostly passing
> now
> > on
> > > > > > branch-1
> > > > > > >>> and master. Lets try and keep it this way and start to trust
> > our
> > > > > test
> > > > > > runs
> > > > > > >>> again. Just a few flakies remain. Lets try and nail them.
> > > > > > >>>
> > > > > > >>> Our tests now run in parallel with other test suites where
> > > previous
> > > > > we
> > > > > > >>> ran alone. You can see this sometimes when our zombie
> detector
> > > > > reports
> > > > > > >>> tests from another project altogether as lingerers (To be
> > fixed).
> > > > > > Some of
> > > > > > >>> our tests are failing because a concurrent hbase run is
> undoing
> > > > > > classes and
> > > > > > >>> data from under it. Also, lets fix.
> > > > > > >>>
> > > > > > >>> Our tests are brittle. It takes 75minutes for them to
> complete.
> > > > Many
> > > > > > >>> are heavy-duty integration tests starting up multiple
> clusters
> > > and
> > > > > > >>> mapreduce all in the one JVM. It is a miracle they pass at
> all.
> > > > > > Usually
> > > > > > >>> integration tests have been cast as unit tests because there
> > was
> > > no
> > > > > > where
> > > > > > >>> else for them to get an airing. We have the hbase-it suite
> now
> > > > which
> > > > > > would
> > > > > > >>> be a more apt place but until these are run on a regular
> basis
> > in
> > > > > > public
> > > > > > >>> for all to see, the fat integration tests disguised as unit
> > tests
> > > > > will
> > > > > > >>> remain. A review of our current unit tests weeding the old
> > cruft
> > > > and
> > > > > > the
> > > > > > >>> no longer relevant or duplicates would be a nice undertaking
> if
> > > > > > someone is
> > > > > > >>> looking to contribute.
> > > > > > >>>
> > > > > > >>> Alex Newman has been working on making our tests work up on
> > > travis
> > > > > and
> > > > > > >>> circle-ci. That'll be sweet when it goes end-to-end. He
> also
> > > > added
> > > > > in
> > > > > > >>> some "type" categorizations -- client, filter, mapreduce --
> > > > alongside
> > > > > > our
> > > > > > >>> old "sizing" categorizations of small/medium/large. His
> > thinking
> > > > is
> > > > > > that
> > > > > > >>> we can run these categorizations in parallel so we could run
> > the
> > > > > total
> > > > > > >>> suite in about the time of the longest test, say
> 20-30minutes?
> > > We
> > > > > > could
> > > > > > >>> even change Apache to run them this way.
> > > > > > >>>
> > > > > > >>> FYI,
> > > > > > >>> St.Ack
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Sean
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Ted Yu <yu...@gmail.com>.
+1 on letting HBase-TRUNK jenkins show coverage report.
Cheers
On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <pa...@gmail.com> wrote:
> Going to change the config of HBase-TRUNK jenkins to show findbugs,
> checkstyle and jacoco coverage report.
> The config has been tested on
> https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30 times. Not
> much different from HBase-TRUNK unless it runs ~30% slower(the overhead of
> collecting information for code coverage).
> Thanks.
>
> 2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>
> > +1, thanks a lot for improving our build hygiene.
> >
> > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> > > Thanks Sean. This is great.
> > >
> > > Enis
> > >
> > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <bu...@cloudera.com>
> > wrote:
> > >
> > > > FYI I just finished chasing down the breakage for mvn site on all
> patch
> > > > builds.
> > > >
> > > > HBASE-13191 consolidates the few places in the test-patch code where
> we
> > > > hard coded MAVEN_OPTS.
> > > >
> > > > If you look at the PreCommit job now, we use the "set environment
> > > > variables" option to set MAVEN_OPTS and then everything else respects
> > > that
> > > > setting.
> > > >
> > > > I set the initial value to be a combination of the memory limitations
> > > we've
> > > > been actually running with (the ~6G was getting ignored) and the
> > permgen
> > > > needed for site.
> > > >
> > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
> > > >
> > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net> wrote:
> > > >
> > > > > I just made TRUNK and branch-1 builds use same jvm as patch-build
> > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be the
> same
> > as
> > > > > those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"... it had
> > been
> > > > > 3000.
> > > > >
> > > > > Yours,
> > > > > St.Ack
> > > > >
> > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net> wrote:
> > > > >
> > > > > > I upped hadoopqa retention to keep last 100 builds and or last 7
> > > days,
> > > > > > whichever comes first.
> > > > > > St.Ack
> > > > > >
> > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net> wrote:
> > > > > >
> > > > > >> Branch-1 and master have stabilized and now run mostly blue
> (give
> > or
> > > > > take
> > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1 has
> helped
> > us
> > > > > >> identify at least one destabilizing commit in the last few days,
> > > maybe
> > > > > two;
> > > > > >> this is as it should be (smile).
> > > > > >>
> > > > > >> Lets keep our builds blue. If you commit a patch, make sure
> > > subsequent
> > > > > >> builds stay blue. You can subscribe to builds@hbase.apache.org
> to
> > > get
> > > > > >> notice of failures if not already subscribed.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> St.Ack
> > > > > >>
> > > > > >> 1. https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> > > > > >> 2.
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> > > > > >>
> > > > > >>
> > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <st...@duboce.net>
> wrote:
> > > > > >>
> > > > > >>> A few notes on testing.
> > > > > >>>
> > > > > >>> Too long to read, infra is more capable now and after some
> work,
> > we
> > > > are
> > > > > >>> seeing branch-1 and trunk mostly running blue. Lets try and
> keep
> > it
> > > > > this
> > > > > >>> way going forward.
> > > > > >>>
> > > > > >>> Apache Infra has new, more capable hardware.
> > > > > >>>
> > > > > >>> A recent spurt of test fixing combined with more capable
> hardware
> > > > seems
> > > > > >>> to have gotten us to a new place; tests are mostly passing now
> on
> > > > > branch-1
> > > > > >>> and master. Lets try and keep it this way and start to trust
> our
> > > > test
> > > > > runs
> > > > > >>> again. Just a few flakies remain. Lets try and nail them.
> > > > > >>>
> > > > > >>> Our tests now run in parallel with other test suites where
> > previous
> > > > we
> > > > > >>> ran alone. You can see this sometimes when our zombie detector
> > > > reports
> > > > > >>> tests from another project altogether as lingerers (To be
> fixed).
> > > > > Some of
> > > > > >>> our tests are failing because a concurrent hbase run is undoing
> > > > > classes and
> > > > > >>> data from under it. Also, lets fix.
> > > > > >>>
> > > > > >>> Our tests are brittle. It takes 75minutes for them to complete.
> > > Many
> > > > > >>> are heavy-duty integration tests starting up multiple clusters
> > and
> > > > > >>> mapreduce all in the one JVM. It is a miracle they pass at all.
> > > > > Usually
> > > > > >>> integration tests have been cast as unit tests because there
> was
> > no
> > > > > where
> > > > > >>> else for them to get an airing. We have the hbase-it suite now
> > > which
> > > > > would
> > > > > >>> be a more apt place but until these are run on a regular basis
> in
> > > > > public
> > > > > >>> for all to see, the fat integration tests disguised as unit
> tests
> > > > will
> > > > > >>> remain. A review of our current unit tests weeding the old
> cruft
> > > and
> > > > > the
> > > > > >>> no longer relevant or duplicates would be a nice undertaking if
> > > > > someone is
> > > > > >>> looking to contribute.
> > > > > >>>
> > > > > >>> Alex Newman has been working on making our tests work up on
> > travis
> > > > and
> > > > > >>> circle-ci. That'll be sweet when it goes end-to-end. He also
> > > added
> > > > in
> > > > > >>> some "type" categorizations -- client, filter, mapreduce --
> > > alongside
> > > > > our
> > > > > >>> old "sizing" categorizations of small/medium/large. His
> thinking
> > > is
> > > > > that
> > > > > >>> we can run these categorizations in parallel so we could run
> the
> > > > total
> > > > > >>> suite in about the time of the longest test, say 20-30minutes?
> > We
> > > > > could
> > > > > >>> even change Apache to run them this way.
> > > > > >>>
> > > > > >>> FYI,
> > > > > >>> St.Ack
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Sean
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by 张铎 <pa...@gmail.com>.
Going to change the config of HBase-TRUNK jenkins to show findbugs,
checkstyle and jacoco coverage report.
The config has been tested on
https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30 times. Not
much different from HBase-TRUNK unless it runs ~30% slower(the overhead of
collecting information for code coverage).
Thanks.
2015-03-12 5:08 GMT+08:00 Andrew Purtell <ap...@apache.org>:
> +1, thanks a lot for improving our build hygiene.
>
> On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <en...@gmail.com> wrote:
>
> > Thanks Sean. This is great.
> >
> > Enis
> >
> > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> > > FYI I just finished chasing down the breakage for mvn site on all patch
> > > builds.
> > >
> > > HBASE-13191 consolidates the few places in the test-patch code where we
> > > hard coded MAVEN_OPTS.
> > >
> > > If you look at the PreCommit job now, we use the "set environment
> > > variables" option to set MAVEN_OPTS and then everything else respects
> > that
> > > setting.
> > >
> > > I set the initial value to be a combination of the memory limitations
> > we've
> > > been actually running with (the ~6G was getting ignored) and the
> permgen
> > > needed for site.
> > >
> > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
> > >
> > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net> wrote:
> > >
> > > > I just made TRUNK and branch-1 builds use same jvm as patch-build
> > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be the same
> as
> > > > those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"... it had
> been
> > > > 3000.
> > > >
> > > > Yours,
> > > > St.Ack
> > > >
> > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net> wrote:
> > > >
> > > > > I upped hadoopqa retention to keep last 100 builds and or last 7
> > days,
> > > > > whichever comes first.
> > > > > St.Ack
> > > > >
> > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net> wrote:
> > > > >
> > > > >> Branch-1 and master have stabilized and now run mostly blue (give
> or
> > > > take
> > > > >> the odd failure) [1][2]. Having a mostly blue branch-1 has helped
> us
> > > > >> identify at least one destabilizing commit in the last few days,
> > maybe
> > > > two;
> > > > >> this is as it should be (smile).
> > > > >>
> > > > >> Lets keep our builds blue. If you commit a patch, make sure
> > subsequent
> > > > >> builds stay blue. You can subscribe to builds@hbase.apache.org to
> > get
> > > > >> notice of failures if not already subscribed.
> > > > >>
> > > > >> Thanks,
> > > > >> St.Ack
> > > > >>
> > > > >> 1. https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> > > > >> 2. https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> > > > >>
> > > > >>
> > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <st...@duboce.net> wrote:
> > > > >>
> > > > >>> A few notes on testing.
> > > > >>>
> > > > >>> Too long to read, infra is more capable now and after some work,
> we
> > > are
> > > > >>> seeing branch-1 and trunk mostly running blue. Lets try and keep
> it
> > > > this
> > > > >>> way going forward.
> > > > >>>
> > > > >>> Apache Infra has new, more capable hardware.
> > > > >>>
> > > > >>> A recent spurt of test fixing combined with more capable hardware
> > > seems
> > > > >>> to have gotten us to a new place; tests are mostly passing now on
> > > > branch-1
> > > > >>> and master. Lets try and keep it this way and start to trust our
> > > test
> > > > runs
> > > > >>> again. Just a few flakies remain. Lets try and nail them.
> > > > >>>
> > > > >>> Our tests now run in parallel with other test suites where
> previous
> > > we
> > > > >>> ran alone. You can see this sometimes when our zombie detector
> > > reports
> > > > >>> tests from another project altogether as lingerers (To be fixed).
> > > > Some of
> > > > >>> our tests are failing because a concurrent hbase run is undoing
> > > > classes and
> > > > >>> data from under it. Also, lets fix.
> > > > >>>
> > > > >>> Our tests are brittle. It takes 75minutes for them to complete.
> > Many
> > > > >>> are heavy-duty integration tests starting up multiple clusters
> and
> > > > >>> mapreduce all in the one JVM. It is a miracle they pass at all.
> > > > Usually
> > > > >>> integration tests have been cast as unit tests because there was
> no
> > > > where
> > > > >>> else for them to get an airing. We have the hbase-it suite now
> > which
> > > > would
> > > > >>> be a more apt place but until these are run on a regular basis in
> > > > public
> > > > >>> for all to see, the fat integration tests disguised as unit tests
> > > will
> > > > >>> remain. A review of our current unit tests weeding the old cruft
> > and
> > > > the
> > > > >>> no longer relevant or duplicates would be a nice undertaking if
> > > > someone is
> > > > >>> looking to contribute.
> > > > >>>
> > > > >>> Alex Newman has been working on making our tests work up on
> travis
> > > and
> > > > >>> circle-ci. That'll be sweet when it goes end-to-end. He also
> > added
> > > in
> > > > >>> some "type" categorizations -- client, filter, mapreduce --
> > alongside
> > > > our
> > > > >>> old "sizing" categorizations of small/medium/large. His thinking
> > is
> > > > that
> > > > >>> we can run these categorizations in parallel so we could run the
> > > total
> > > > >>> suite in about the time of the longest test, say 20-30minutes?
> We
> > > > could
> > > > >>> even change Apache to run them this way.
> > > > >>>
> > > > >>> FYI,
> > > > >>> St.Ack
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Sean
> > >
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Andrew Purtell <ap...@apache.org>.
+1, thanks a lot for improving our build hygiene.
On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <en...@gmail.com> wrote:
> Thanks Sean. This is great.
>
> Enis
>
> On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
> > FYI I just finished chasing down the breakage for mvn site on all patch
> > builds.
> >
> > HBASE-13191 consolidates the few places in the test-patch code where we
> > hard coded MAVEN_OPTS.
> >
> > If you look at the PreCommit job now, we use the "set environment
> > variables" option to set MAVEN_OPTS and then everything else respects
> that
> > setting.
> >
> > I set the initial value to be a combination of the memory limitations
> we've
> > been actually running with (the ~6G was getting ignored) and the permgen
> > needed for site.
> >
> > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
> >
> > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net> wrote:
> >
> > > I just made TRUNK and branch-1 builds use same jvm as patch-build
> > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be the same as
> > > those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"... it had been
> > > 3000.
> > >
> > > Yours,
> > > St.Ack
> > >
> > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net> wrote:
> > >
> > > > I upped hadoopqa retention to keep last 100 builds and or last 7
> days,
> > > > whichever comes first.
> > > > St.Ack
> > > >
> > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net> wrote:
> > > >
> > > >> Branch-1 and master have stabilized and now run mostly blue (give or
> > > take
> > > >> the odd failure) [1][2]. Having a mostly blue branch-1 has helped us
> > > >> identify at least one destabilizing commit in the last few days,
> maybe
> > > two;
> > > >> this is as it should be (smile).
> > > >>
> > > >> Lets keep our builds blue. If you commit a patch, make sure
> subsequent
> > > >> builds stay blue. You can subscribe to builds@hbase.apache.org to
> get
> > > >> notice of failures if not already subscribed.
> > > >>
> > > >> Thanks,
> > > >> St.Ack
> > > >>
> > > >> 1. https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> > > >> 2. https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> > > >>
> > > >>
> > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <st...@duboce.net> wrote:
> > > >>
> > > >>> A few notes on testing.
> > > >>>
> > > >>> Too long to read, infra is more capable now and after some work, we
> > are
> > > >>> seeing branch-1 and trunk mostly running blue. Lets try and keep it
> > > this
> > > >>> way going forward.
> > > >>>
> > > >>> Apache Infra has new, more capable hardware.
> > > >>>
> > > >>> A recent spurt of test fixing combined with more capable hardware
> > seems
> > > >>> to have gotten us to a new place; tests are mostly passing now on
> > > branch-1
> > > >>> and master. Lets try and keep it this way and start to trust our
> > test
> > > runs
> > > >>> again. Just a few flakies remain. Lets try and nail them.
> > > >>>
> > > >>> Our tests now run in parallel with other test suites where previous
> > we
> > > >>> ran alone. You can see this sometimes when our zombie detector
> > reports
> > > >>> tests from another project altogether as lingerers (To be fixed).
> > > Some of
> > > >>> our tests are failing because a concurrent hbase run is undoing
> > > classes and
> > > >>> data from under it. Also, lets fix.
> > > >>>
> > > >>> Our tests are brittle. It takes 75minutes for them to complete.
> Many
> > > >>> are heavy-duty integration tests starting up multiple clusters and
> > > >>> mapreduce all in the one JVM. It is a miracle they pass at all.
> > > Usually
> > > >>> integration tests have been cast as unit tests because there was no
> > > where
> > > >>> else for them to get an airing. We have the hbase-it suite now
> which
> > > would
> > > >>> be a more apt place but until these are run on a regular basis in
> > > public
> > > >>> for all to see, the fat integration tests disguised as unit tests
> > will
> > > >>> remain. A review of our current unit tests weeding the old cruft
> and
> > > the
> > > >>> no longer relevant or duplicates would be a nice undertaking if
> > > someone is
> > > >>> looking to contribute.
> > > >>>
> > > >>> Alex Newman has been working on making our tests work up on travis
> > and
> > > >>> circle-ci. That'll be sweet when it goes end-to-end. He also
> added
> > in
> > > >>> some "type" categorizations -- client, filter, mapreduce --
> alongside
> > > our
> > > >>> old "sizing" categorizations of small/medium/large. His thinking
> is
> > > that
> > > >>> we can run these categorizations in parallel so we could run the
> > total
> > > >>> suite in about the time of the longest test, say 20-30minutes? We
> > > could
> > > >>> even change Apache to run them this way.
> > > >>>
> > > >>> FYI,
> > > >>> St.Ack
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > > >
> > >
> >
> >
> >
> > --
> > Sean
> >
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Enis Söztutar <en...@gmail.com>.
Thanks Sean. This is great.
Enis
On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <bu...@cloudera.com> wrote:
> FYI I just finished chasing down the breakage for mvn site on all patch
> builds.
>
> HBASE-13191 consolidates the few places in the test-patch code where we
> hard coded MAVEN_OPTS.
>
> If you look at the PreCommit job now, we use the "set environment
> variables" option to set MAVEN_OPTS and then everything else respects that
> setting.
>
> I set the initial value to be a combination of the memory limitations we've
> been actually running with (the ~6G was getting ignored) and the permgen
> needed for site.
>
> MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
>
> On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net> wrote:
>
> > I just made TRUNK and branch-1 builds use same jvm as patch-build
> > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be the same as
> > those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"... it had been
> > 3000.
> >
> > Yours,
> > St.Ack
> >
> > On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net> wrote:
> >
> > > I upped hadoopqa retention to keep last 100 builds and or last 7 days,
> > > whichever comes first.
> > > St.Ack
> > >
> > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net> wrote:
> > >
> > >> Branch-1 and master have stabilized and now run mostly blue (give or
> > take
> > >> the odd failure) [1][2]. Having a mostly blue branch-1 has helped us
> > >> identify at least one destabilizing commit in the last few days, maybe
> > two;
> > >> this is as it should be (smile).
> > >>
> > >> Lets keep our builds blue. If you commit a patch, make sure subsequent
> > >> builds stay blue. You can subscribe to builds@hbase.apache.org to get
> > >> notice of failures if not already subscribed.
> > >>
> > >> Thanks,
> > >> St.Ack
> > >>
> > >> 1. https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> > >> 2. https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> > >>
> > >>
> > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <st...@duboce.net> wrote:
> > >>
> > >>> A few notes on testing.
> > >>>
> > >>> Too long to read, infra is more capable now and after some work, we
> are
> > >>> seeing branch-1 and trunk mostly running blue. Lets try and keep it
> > this
> > >>> way going forward.
> > >>>
> > >>> Apache Infra has new, more capable hardware.
> > >>>
> > >>> A recent spurt of test fixing combined with more capable hardware
> seems
> > >>> to have gotten us to a new place; tests are mostly passing now on
> > branch-1
> > >>> and master. Lets try and keep it this way and start to trust our
> test
> > runs
> > >>> again. Just a few flakies remain. Lets try and nail them.
> > >>>
> > >>> Our tests now run in parallel with other test suites where previous
> we
> > >>> ran alone. You can see this sometimes when our zombie detector
> reports
> > >>> tests from another project altogether as lingerers (To be fixed).
> > Some of
> > >>> our tests are failing because a concurrent hbase run is undoing
> > classes and
> > >>> data from under it. Also, lets fix.
> > >>>
> > >>> Our tests are brittle. It takes 75minutes for them to complete. Many
> > >>> are heavy-duty integration tests starting up multiple clusters and
> > >>> mapreduce all in the one JVM. It is a miracle they pass at all.
> > Usually
> > >>> integration tests have been cast as unit tests because there was no
> > where
> > >>> else for them to get an airing. We have the hbase-it suite now which
> > would
> > >>> be a more apt place but until these are run on a regular basis in
> > public
> > >>> for all to see, the fat integration tests disguised as unit tests
> will
> > >>> remain. A review of our current unit tests weeding the old cruft and
> > the
> > >>> no longer relevant or duplicates would be a nice undertaking if
> > someone is
> > >>> looking to contribute.
> > >>>
> > >>> Alex Newman has been working on making our tests work up on travis
> and
> > >>> circle-ci. That'll be sweet when it goes end-to-end. He also added
> in
> > >>> some "type" categorizations -- client, filter, mapreduce -- alongside
> > our
> > >>> old "sizing" categorizations of small/medium/large. His thinking is
> > that
> > >>> we can run these categorizations in parallel so we could run the
> total
> > >>> suite in about the time of the longest test, say 20-30minutes? We
> > could
> > >>> even change Apache to run them this way.
> > >>>
> > >>> FYI,
> > >>> St.Ack
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>
> > >
> >
>
>
>
> --
> Sean
>
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Sean Busbey <bu...@cloudera.com>.
FYI I just finished chasing down the breakage for mvn site on all patch
builds.
HBASE-13191 consolidates the few places in the test-patch code where we
hard coded MAVEN_OPTS.
If you look at the PreCommit job now, we use the "set environment
variables" option to set MAVEN_OPTS and then everything else respects that
setting.
I set the initial value to be a combination of the memory limitations we've
been actually running with (the ~6G was getting ignored) and the permgen
needed for site.
MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net> wrote:
> I just made TRUNK and branch-1 builds use same jvm as patch-build
> (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be the same as
> those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"... it had been
> 3000.
>
> Yours,
> St.Ack
>
> On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net> wrote:
>
> > I upped hadoopqa retention to keep last 100 builds and or last 7 days,
> > whichever comes first.
> > St.Ack
> >
> > On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net> wrote:
> >
> >> Branch-1 and master have stabilized and now run mostly blue (give or
> take
> >> the odd failure) [1][2]. Having a mostly blue branch-1 has helped us
> >> identify at least one destabilizing commit in the last few days, maybe
> two;
> >> this is as it should be (smile).
> >>
> >> Lets keep our builds blue. If you commit a patch, make sure subsequent
> >> builds stay blue. You can subscribe to builds@hbase.apache.org to get
> >> notice of failures if not already subscribed.
> >>
> >> Thanks,
> >> St.Ack
> >>
> >> 1. https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> >> 2. https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> >>
> >>
> >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <st...@duboce.net> wrote:
> >>
> >>> A few notes on testing.
> >>>
> >>> Too long to read, infra is more capable now and after some work, we are
> >>> seeing branch-1 and trunk mostly running blue. Lets try and keep it
> this
> >>> way going forward.
> >>>
> >>> Apache Infra has new, more capable hardware.
> >>>
> >>> A recent spurt of test fixing combined with more capable hardware seems
> >>> to have gotten us to a new place; tests are mostly passing now on
> branch-1
> >>> and master. Lets try and keep it this way and start to trust our test
> runs
> >>> again. Just a few flakies remain. Lets try and nail them.
> >>>
> >>> Our tests now run in parallel with other test suites where previous we
> >>> ran alone. You can see this sometimes when our zombie detector reports
> >>> tests from another project altogether as lingerers (To be fixed).
> Some of
> >>> our tests are failing because a concurrent hbase run is undoing
> classes and
> >>> data from under it. Also, lets fix.
> >>>
> >>> Our tests are brittle. It takes 75minutes for them to complete. Many
> >>> are heavy-duty integration tests starting up multiple clusters and
> >>> mapreduce all in the one JVM. It is a miracle they pass at all.
> Usually
> >>> integration tests have been cast as unit tests because there was no
> where
> >>> else for them to get an airing. We have the hbase-it suite now which
> would
> >>> be a more apt place but until these are run on a regular basis in
> public
> >>> for all to see, the fat integration tests disguised as unit tests will
> >>> remain. A review of our current unit tests weeding the old cruft and
> the
> >>> no longer relevant or duplicates would be a nice undertaking if
> someone is
> >>> looking to contribute.
> >>>
> >>> Alex Newman has been working on making our tests work up on travis and
> >>> circle-ci. That'll be sweet when it goes end-to-end. He also added in
> >>> some "type" categorizations -- client, filter, mapreduce -- alongside
> our
> >>> old "sizing" categorizations of small/medium/large. His thinking is
> that
> >>> we can run these categorizations in parallel so we could run the total
> >>> suite in about the time of the longest test, say 20-30minutes? We
> could
> >>> even change Apache to run them this way.
> >>>
> >>> FYI,
> >>> St.Ack
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >
>
--
Sean
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Posted by Stack <st...@duboce.net>.
I just made TRUNK and branch-1 builds use same jvm as patch-build
(hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be the same as
those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"... it had been
3000.
Yours,
St.Ack
On Wed, Dec 31, 2014 at 4:22 PM, Stack <st...@duboce.net> wrote:
> I upped hadoopqa retention to keep last 100 builds and or last 7 days,
> whichever comes first.
> St.Ack
>
> On Tue, Nov 4, 2014 at 9:38 AM, Stack <st...@duboce.net> wrote:
>
>> Branch-1 and master have stabilized and now run mostly blue (give or take
>> the odd failure) [1][2]. Having a mostly blue branch-1 has helped us
>> identify at least one destabilizing commit in the last few days, maybe two;
>> this is as it should be (smile).
>>
>> Lets keep our builds blue. If you commit a patch, make sure subsequent
>> builds stay blue. You can subscribe to builds@hbase.apache.org to get
>> notice of failures if not already subscribed.
>>
>> Thanks,
>> St.Ack
>>
>> 1. https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>> 2. https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>>
>>
>> On Mon, Oct 13, 2014 at 4:41 PM, Stack <st...@duboce.net> wrote:
>>
>>> A few notes on testing.
>>>
>>> Too long to read, infra is more capable now and after some work, we are
>>> seeing branch-1 and trunk mostly running blue. Lets try and keep it this
>>> way going forward.
>>>
>>> Apache Infra has new, more capable hardware.
>>>
>>> A recent spurt of test fixing combined with more capable hardware seems
>>> to have gotten us to a new place; tests are mostly passing now on branch-1
>>> and master. Lets try and keep it this way and start to trust our test runs
>>> again. Just a few flakies remain. Lets try and nail them.
>>>
>>> Our tests now run in parallel with other test suites where previous we
>>> ran alone. You can see this sometimes when our zombie detector reports
>>> tests from another project altogether as lingerers (To be fixed). Some of
>>> our tests are failing because a concurrent hbase run is undoing classes and
>>> data from under it. Also, lets fix.
>>>
>>> Our tests are brittle. It takes 75minutes for them to complete. Many
>>> are heavy-duty integration tests starting up multiple clusters and
>>> mapreduce all in the one JVM. It is a miracle they pass at all. Usually
>>> integration tests have been cast as unit tests because there was no where
>>> else for them to get an airing. We have the hbase-it suite now which would
>>> be a more apt place but until these are run on a regular basis in public
>>> for all to see, the fat integration tests disguised as unit tests will
>>> remain. A review of our current unit tests weeding the old cruft and the
>>> no longer relevant or duplicates would be a nice undertaking if someone is
>>> looking to contribute.
>>>
>>> Alex Newman has been working on making our tests work up on travis and
>>> circle-ci. That'll be sweet when it goes end-to-end. He also added in
>>> some "type" categorizations -- client, filter, mapreduce -- alongside our
>>> old "sizing" categorizations of small/medium/large. His thinking is that
>>> we can run these categorizations in parallel so we could run the total
>>> suite in about the time of the longest test, say 20-30minutes? We could
>>> even change Apache to run them this way.
>>>
>>> FYI,
>>> St.Ack
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>