You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Clebert Suconic <cl...@gmail.com> on 2016/07/05 03:42:06 UTC
Re: How do I compile and run Artemis tests?
Did you manage to do it. Or still have issues ?
On Tuesday, June 28, 2016, Jiri Danek <jd...@redhat.com> wrote:
> After somewhat lengthy struggle I still did not manage to run Artemis tests
> in a VM. I'd like to ask if there is anybody who managed to run the tests
> and I would like to ask for suggestions what could be causing the tests to
> fail for me. Is there something obvious I need to do but I am not doing?
>
> I am running this series of commands
>
> mvn -Prelease clean package install
> mvn -Ptests test
>
> My issues to date were ... mostly hardware requirements.
>
> First of all, I checked the source on Saturday when the build was broken,
> so I had to checkout 1.3.0 tag.
>
> Running the tests seems to require 4GB of RAM (tried 512MB, then 2GB, and
> that did not work) and high open file descriptor limit, the default 1024 is
> not enough, I set 40 000, because that seems to be what the Apache Jenkins
> slaves use. It also requires a lot of time, because the tests run long.
>
> The settings above got me probably furthest I ever got. Before I increased
> the descriptor limit, tests got stuck at repeatedly printing
>
> [main] 11:42:52,381 WARN [org.apache.activemq.artemis.core.client]
> AMQ212054: Destination address=FailoverTestAddress is blocked. If the
> system is configured to block make sure you consume messages on this
> configuration.
> [main] 11:43:02,381 WARN [org.apache.activemq.artemis.core.client]
> AMQ212054: Destination address=FailoverTestAddress is blocked. If the
> system is configured to block make sure you consume messages on this
> configuration.
> [main] 11:43:12,381 WARN [org.apache.activemq.artemis.core.client]
> AMQ212054: Destination address=FailoverTestAddress is blocked. If the
> system is configured to block make sure you consume messages on this
> configuration.
> [main] 11:43:22,381 WARN [org.apache.activemq.artemis.core.client]
> AMQ212054: Destination address=FailoverTestAddress is blocked. If the
> system is configured to block make sure you consume messages on this
> configuration.
>
> After increasing the limit, I got stuck at a test repeatedly printing
>
> javax.jms.IllegalStateException: The Session is closed
> at
> org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:775)
> at
>
> org.apache.activemq.ActiveMQSession.configureMessage(ActiveMQSession.java:763)
> at
>
> org.apache.activemq.ActiveMQSession.createTextMessage(ActiveMQSession.java:440)
> at
>
> org.apache.activemq.artemis.tests.integration.openwire.amq.ProducerFlowControlSendFailTest$1.run(ProducerFlowControlSendFailTest.java:96)
>
> But I do not know how to move from here. Thanks for any input,
> --
> Jiří Daněk
>
--
Clebert Suconic
Re: How do I compile and run Artemis tests?
Posted by Clebert Suconic <cl...@gmail.com>.
These Leaking threads are normal options...
The tests are using org.apache.activemq.artemis.tests.util.ThreadLeakCheckRule.
Since my early days in Messaging a common issue on running tests is a
leaking server bumping into the next test.
To solve this I had written checks to validate that the next test will
run clean. But sometimes an executor will take longer to have its
thread dying and there is a retry on the ThreadLeakCheckRule, at the
end you should see cleared after retries, and that should be fine. But
I keep these reported in case there's something else going on.
On Wed, Jul 6, 2016 at 6:34 PM, Jiri Danek <jd...@redhat.com> wrote:
> On Tue, Jul 5, 2016 at 5:01 PM, Clebert Suconic <cl...@gmail.com>
> wrote:
>>
>> This test is currently failing, but I didn't see it hanging anywhere.
>>
>>
>> Perhaps we should add it to the ignore list until it's fixed.
>
>
> I am attaching my output of `mvn -Pfast-tests -Pextra-tests install`. It
> looks much different than what Jenkins at Apache produces for the same
> commit,
> https://builds.apache.org/job/ActiveMQ-Artemis-PR-Build/1446/consoleFull
>
> Could you please take look at it? Maybe you could guess why is my run so
> much different. For me, three tests fail, whereas on Jenkins everything
> succeeds. Also, in my run, there are scary messages like
>
> *********************************************************************************
> LEAKING THREADS
> =============================================================================
> Thread Thread[Thread-0
> (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@445b295b-1239807799),5,ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@445b295b-1239807799]
> is still alive with the following stackTrace:
> sun.misc.Unsafe.unpark(Native Method)
> java.util.concurrent.locks.LockSupport.unpark(LockSupport.java:141)
> java.util.concurrent.locks.AbstractQueuedSynchronizer.unparkSuccessor(AbstractQueuedSynchronizer.java:662)
> java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1264)
> java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
> java.util.concurrent.ThreadPoolExecutor.tryTerminate(ThreadPoolExecutor.java:714)
> java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1007)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> *********************************************************************************
> [main] 20:06:48,806 INFO [org.apache.activemq.artemis.core.server] #test
> forceGC
> [main] 20:06:48,831 INFO [org.apache.activemq.artemis.core.server] #test
> forceGC Done
> ******************** Threads cleared after retries ********************
>
> Thanks for any help,
> --
> Jiří Daněk
>
--
Clebert Suconic
Re: How do I compile and run Artemis tests?
Posted by Clebert Suconic <cl...@gmail.com>.
do you have any patches you are working on? what else should I look for?
On Thu, Jul 7, 2016 at 2:18 PM, Clebert Suconic
<cl...@gmail.com> wrote:
> What else should I look for?
>
> On Wed, Jul 6, 2016 at 6:34 PM, Jiri Danek <jd...@redhat.com> wrote:
>> On Tue, Jul 5, 2016 at 5:01 PM, Clebert Suconic <cl...@gmail.com>
>> wrote:
>>>
>>> This test is currently failing, but I didn't see it hanging anywhere.
>>>
>>>
>>> Perhaps we should add it to the ignore list until it's fixed.
>>
>>
>> I am attaching my output of `mvn -Pfast-tests -Pextra-tests install`. It
>> looks much different than what Jenkins at Apache produces for the same
>> commit,
>> https://builds.apache.org/job/ActiveMQ-Artemis-PR-Build/1446/consoleFull
>>
>> Could you please take look at it? Maybe you could guess why is my run so
>> much different. For me, three tests fail, whereas on Jenkins everything
>> succeeds. Also, in my run, there are scary messages like
>>
>> *********************************************************************************
>> LEAKING THREADS
>> =============================================================================
>> Thread Thread[Thread-0
>> (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@445b295b-1239807799),5,ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@445b295b-1239807799]
>> is still alive with the following stackTrace:
>> sun.misc.Unsafe.unpark(Native Method)
>> java.util.concurrent.locks.LockSupport.unpark(LockSupport.java:141)
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.unparkSuccessor(AbstractQueuedSynchronizer.java:662)
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1264)
>> java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
>> java.util.concurrent.ThreadPoolExecutor.tryTerminate(ThreadPoolExecutor.java:714)
>> java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1007)
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> java.lang.Thread.run(Thread.java:745)
>> *********************************************************************************
>> [main] 20:06:48,806 INFO [org.apache.activemq.artemis.core.server] #test
>> forceGC
>> [main] 20:06:48,831 INFO [org.apache.activemq.artemis.core.server] #test
>> forceGC Done
>> ******************** Threads cleared after retries ********************
>>
>> Thanks for any help,
>> --
>> Jiří Daněk
>>
>
>
>
> --
> Clebert Suconic
--
Clebert Suconic
Re: How do I compile and run Artemis tests?
Posted by Clebert Suconic <cl...@gmail.com>.
What else should I look for?
On Wed, Jul 6, 2016 at 6:34 PM, Jiri Danek <jd...@redhat.com> wrote:
> On Tue, Jul 5, 2016 at 5:01 PM, Clebert Suconic <cl...@gmail.com>
> wrote:
>>
>> This test is currently failing, but I didn't see it hanging anywhere.
>>
>>
>> Perhaps we should add it to the ignore list until it's fixed.
>
>
> I am attaching my output of `mvn -Pfast-tests -Pextra-tests install`. It
> looks much different than what Jenkins at Apache produces for the same
> commit,
> https://builds.apache.org/job/ActiveMQ-Artemis-PR-Build/1446/consoleFull
>
> Could you please take look at it? Maybe you could guess why is my run so
> much different. For me, three tests fail, whereas on Jenkins everything
> succeeds. Also, in my run, there are scary messages like
>
> *********************************************************************************
> LEAKING THREADS
> =============================================================================
> Thread Thread[Thread-0
> (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@445b295b-1239807799),5,ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@445b295b-1239807799]
> is still alive with the following stackTrace:
> sun.misc.Unsafe.unpark(Native Method)
> java.util.concurrent.locks.LockSupport.unpark(LockSupport.java:141)
> java.util.concurrent.locks.AbstractQueuedSynchronizer.unparkSuccessor(AbstractQueuedSynchronizer.java:662)
> java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1264)
> java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
> java.util.concurrent.ThreadPoolExecutor.tryTerminate(ThreadPoolExecutor.java:714)
> java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1007)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> *********************************************************************************
> [main] 20:06:48,806 INFO [org.apache.activemq.artemis.core.server] #test
> forceGC
> [main] 20:06:48,831 INFO [org.apache.activemq.artemis.core.server] #test
> forceGC Done
> ******************** Threads cleared after retries ********************
>
> Thanks for any help,
> --
> Jiří Daněk
>
--
Clebert Suconic
Re: How do I compile and run Artemis tests?
Posted by Jiri Danek <jd...@redhat.com>.
On Tue, Jul 5, 2016 at 5:01 PM, Clebert Suconic <cl...@gmail.com>
wrote:
> This test is currently failing, but I didn't see it hanging anywhere.
>
>
> Perhaps we should add it to the ignore list until it's fixed.
>
I am attaching my output of `mvn -Pfast-tests -Pextra-tests install`. It
looks much different than what Jenkins at Apache produces for the same
commit,
https://builds.apache.org/job/ActiveMQ-Artemis-PR-Build/1446/consoleFull
Could you please take look at it? Maybe you could guess why is my run so
much different. For me, three tests fail, whereas on Jenkins everything
succeeds. Also, in my run, there are scary messages like
*********************************************************************************
LEAKING THREADS
=============================================================================
Thread Thread[Thread-0
(ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@445b295b-1239807799
),5,ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@445b295b-1239807799]
is still alive with the following stackTrace:
sun.misc.Unsafe.unpark(Native Method)
java.util.concurrent.locks.LockSupport.unpark(LockSupport.java:141)
java.util.concurrent.locks.AbstractQueuedSynchronizer.unparkSuccessor(AbstractQueuedSynchronizer.java:662)
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1264)
java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
java.util.concurrent.ThreadPoolExecutor.tryTerminate(ThreadPoolExecutor.java:714)
java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1007)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
*********************************************************************************
[main] 20:06:48,806 INFO [org.apache.activemq.artemis.core.server] #test
forceGC
[main] 20:06:48,831 INFO [org.apache.activemq.artemis.core.server] #test
forceGC Done
******************** Threads cleared after retries ********************
Thanks for any help,
--
Jiří Daněk
Re: How do I compile and run Artemis tests?
Posted by Clebert Suconic <cl...@gmail.com>.
This test is currently failing, but I didn't see it hanging anywhere.
Perhaps we should add it to the ignore list until it's fixed.
On Tue, Jul 5, 2016 at 9:27 AM, Jiri Danek <jd...@redhat.com> wrote:
> On Tuesday, June 28, 2016, Jiri Danek <jd...@redhat.com> wrote:
>
>> After increasing the [memory] limit, I got stuck at a test repeatedly
>> printing
>>
>> javax.jms.IllegalStateException: The Session is closed
>> at
>> org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:775)
>> at
>> org.apache.activemq.ActiveMQSession.configureMessage(ActiveMQSession.java:763)
>> at
>> org.apache.activemq.ActiveMQSession.createTextMessage(ActiveMQSession.java:440)
>> at
>> org.apache.activemq.artemis.tests.integration.openwire.amq.ProducerFlowControlSendFailTest$1.run(ProducerFlowControlSendFailTest.java:96)
>>
>
> On Tue, Jul 5, 2016 at 5:42 AM, Clebert Suconic <cl...@gmail.com>
> wrote:
>
>> Did you manage to do it. Or still have issues ?
>>
>
> No, I did not manage to get over the "ProducerFlowControlSendFailTest"
> issue I mention above. I tried giving the VM even more memory, 8GB, but
> that did not help. Any suggestions what I should try?
>
> --
> Jiří Daněk
--
Clebert Suconic
Re: How do I compile and run Artemis tests?
Posted by Jiri Danek <jd...@redhat.com>.
On Tuesday, June 28, 2016, Jiri Danek <jd...@redhat.com> wrote:
> After increasing the [memory] limit, I got stuck at a test repeatedly
> printing
>
> javax.jms.IllegalStateException: The Session is closed
> at
> org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:775)
> at
> org.apache.activemq.ActiveMQSession.configureMessage(ActiveMQSession.java:763)
> at
> org.apache.activemq.ActiveMQSession.createTextMessage(ActiveMQSession.java:440)
> at
> org.apache.activemq.artemis.tests.integration.openwire.amq.ProducerFlowControlSendFailTest$1.run(ProducerFlowControlSendFailTest.java:96)
>
On Tue, Jul 5, 2016 at 5:42 AM, Clebert Suconic <cl...@gmail.com>
wrote:
> Did you manage to do it. Or still have issues ?
>
No, I did not manage to get over the "ProducerFlowControlSendFailTest"
issue I mention above. I tried giving the VM even more memory, 8GB, but
that did not help. Any suggestions what I should try?
--
Jiří Daněk