You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by Kevin Burton <bu...@spinn3r.com> on 2015/04/01 21:48:25 UTC
How is ActiveMQ 6.0 for continual integration / testing?
This was one of the things that bothered/bothers me about ActiveMQ 5.x… the
test takes 24 hours to run. I’ve never actually had any success getting it
to run because it’s crashed on me every time. (even for release builds).
Our internal tests take 20 minutes to build… and that is too long IMO.
Certainly our tests could be more comprehensive but I think they’re pretty
damn good right now.
Would be good if ActiveMQ 6.0 could resolve this and we could get builds
integrated quickly after a commit. :)
Kevin
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: How is ActiveMQ 6.0 for continual integration / testing?
Posted by Clebert Suconic <cl...@gmail.com>.
it should take 1 hour and half. Maybe we could fix some issues on the
tests and make it to 1 hour.
Up to now we have been taking good care on avoiding sleeps and making
sure the tests run without delays. There are a few tests that will
need to validate expiration and etc so it's unavoidable a few sleeps
(like failover for instance).
A lot of time on the testsuite is being spent on Failover and Bridge
Tests.. we could make them running faster without losing coverage.
There is also the stress-testsuite and soak tests but those are
basically an extension of the testsuite (increasing number of
iterations in some of the tests); we will eventually make that part of
the regression builds.. but we should make that less frequent (once a
week and always before releases).
The testsuite now is not working as I broke the testsuite in some of
my last commits (I'm trying to fix that now), but this should give you
an idea of what we run:
https://builds.apache.org/view/All/job/ActiveMQ6-Nightly-Regression-Test/68/testReport/
I should have it fixed after the holidays (working on it)
On Thu, Apr 2, 2015 at 9:22 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> NICE. Also, how long does the entire test / regression suite take? …
> roughly.
>
> On Wed, Apr 1, 2015 at 1:01 PM, Clebert Suconic <cl...@gmail.com>
> wrote:
>
>> For every Pull Request sent (e.g:
>> https://github.com/apache/activemq-6/pull/197)
>>
>> There will be a build with a considerable sized testsuite running..
>> which we call fast-test
>> https://builds.apache.org/job/ActiveMQ6-PR-Build/223/
>>
>>
>> This will validate tests, CheckStyle and Licensing.
>>
>>
>>
>> This will run the entire JMS tests, unit-tests and some other tests.
>> It's worth looking at:
>> https://builds.apache.org/job/ActiveMQ6-PR-Build/223/testReport/
>>
>>
>>
>> We then run the entire testsuite daily:
>> https://builds.apache.org/view/All/job/ActiveMQ6-Nightly-Regression-Test/
>>
>>
>> There are some failures on this environment now but that's something
>> we are working to solve with Infra. They run ok on my personal dev
>> enrionment and some other servers I have access to.
>>
>> We are in hold now based on the discussions on dev-list this week, but
>> we should fix that soon.
>>
>>
>> Anyway, I hope you get the picture.. PRs are rejected if they fail the
>> basic testsuite, and we will monitor the entire testsuite daily.
>>
>> On Wed, Apr 1, 2015 at 3:48 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>> > This was one of the things that bothered/bothers me about ActiveMQ 5.x…
>> the
>> > test takes 24 hours to run. I’ve never actually had any success getting
>> it
>> > to run because it’s crashed on me every time. (even for release builds).
>> >
>> > Our internal tests take 20 minutes to build… and that is too long IMO.
>> > Certainly our tests could be more comprehensive but I think they’re
>> pretty
>> > damn good right now.
>> >
>> > Would be good if ActiveMQ 6.0 could resolve this and we could get builds
>> > integrated quickly after a commit. :)
>> >
>> > Kevin
>> >
>> > --
>> >
>> > Founder/CEO Spinn3r.com
>> > Location: *San Francisco, CA*
>> > blog: http://burtonator.wordpress.com
>> > … or check out my Google+ profile
>> > <https://plus.google.com/102718274791889610666/posts>
>> > <http://spinn3r.com>
>>
>>
>>
>> --
>> Clebert Suconic
>> http://community.jboss.org/people/clebert.suconic@jboss.com
>> http://clebertsuconic.blogspot.com
>>
>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
--
Clebert Suconic
http://community.jboss.org/people/clebert.suconic@jboss.com
http://clebertsuconic.blogspot.com
Re: How is ActiveMQ 6.0 for continual integration / testing?
Posted by Kevin Burton <bu...@spinn3r.com>.
NICE. Also, how long does the entire test / regression suite take? …
roughly.
On Wed, Apr 1, 2015 at 1:01 PM, Clebert Suconic <cl...@gmail.com>
wrote:
> For every Pull Request sent (e.g:
> https://github.com/apache/activemq-6/pull/197)
>
> There will be a build with a considerable sized testsuite running..
> which we call fast-test
> https://builds.apache.org/job/ActiveMQ6-PR-Build/223/
>
>
> This will validate tests, CheckStyle and Licensing.
>
>
>
> This will run the entire JMS tests, unit-tests and some other tests.
> It's worth looking at:
> https://builds.apache.org/job/ActiveMQ6-PR-Build/223/testReport/
>
>
>
> We then run the entire testsuite daily:
> https://builds.apache.org/view/All/job/ActiveMQ6-Nightly-Regression-Test/
>
>
> There are some failures on this environment now but that's something
> we are working to solve with Infra. They run ok on my personal dev
> enrionment and some other servers I have access to.
>
> We are in hold now based on the discussions on dev-list this week, but
> we should fix that soon.
>
>
> Anyway, I hope you get the picture.. PRs are rejected if they fail the
> basic testsuite, and we will monitor the entire testsuite daily.
>
> On Wed, Apr 1, 2015 at 3:48 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> > This was one of the things that bothered/bothers me about ActiveMQ 5.x…
> the
> > test takes 24 hours to run. I’ve never actually had any success getting
> it
> > to run because it’s crashed on me every time. (even for release builds).
> >
> > Our internal tests take 20 minutes to build… and that is too long IMO.
> > Certainly our tests could be more comprehensive but I think they’re
> pretty
> > damn good right now.
> >
> > Would be good if ActiveMQ 6.0 could resolve this and we could get builds
> > integrated quickly after a commit. :)
> >
> > Kevin
> >
> > --
> >
> > Founder/CEO Spinn3r.com
> > Location: *San Francisco, CA*
> > blog: http://burtonator.wordpress.com
> > … or check out my Google+ profile
> > <https://plus.google.com/102718274791889610666/posts>
> > <http://spinn3r.com>
>
>
>
> --
> Clebert Suconic
> http://community.jboss.org/people/clebert.suconic@jboss.com
> http://clebertsuconic.blogspot.com
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: How is ActiveMQ 6.0 for continual integration / testing?
Posted by Clebert Suconic <cl...@gmail.com>.
For every Pull Request sent (e.g: https://github.com/apache/activemq-6/pull/197)
There will be a build with a considerable sized testsuite running..
which we call fast-test
https://builds.apache.org/job/ActiveMQ6-PR-Build/223/
This will validate tests, CheckStyle and Licensing.
This will run the entire JMS tests, unit-tests and some other tests.
It's worth looking at:
https://builds.apache.org/job/ActiveMQ6-PR-Build/223/testReport/
We then run the entire testsuite daily:
https://builds.apache.org/view/All/job/ActiveMQ6-Nightly-Regression-Test/
There are some failures on this environment now but that's something
we are working to solve with Infra. They run ok on my personal dev
enrionment and some other servers I have access to.
We are in hold now based on the discussions on dev-list this week, but
we should fix that soon.
Anyway, I hope you get the picture.. PRs are rejected if they fail the
basic testsuite, and we will monitor the entire testsuite daily.
On Wed, Apr 1, 2015 at 3:48 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> This was one of the things that bothered/bothers me about ActiveMQ 5.x… the
> test takes 24 hours to run. I’ve never actually had any success getting it
> to run because it’s crashed on me every time. (even for release builds).
>
> Our internal tests take 20 minutes to build… and that is too long IMO.
> Certainly our tests could be more comprehensive but I think they’re pretty
> damn good right now.
>
> Would be good if ActiveMQ 6.0 could resolve this and we could get builds
> integrated quickly after a commit. :)
>
> Kevin
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
--
Clebert Suconic
http://community.jboss.org/people/clebert.suconic@jboss.com
http://clebertsuconic.blogspot.com