You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by jnicolay <j....@topdesk.com> on 2016/07/20 15:15:31 UTC
problems getting test results deterministic
Hi all
We ran into some strange behavior in our application and I tried to reproduce
it in a unit test.
Our setup looks somewhat like this:
+-----------+ +--------+ +-----------+
| embedded1 | <---> | master | <--> | embedded2 |
+-----------+ +--------+ +-----------+
\------ ----/ \--- ------/
\/ \/
+-----------+ | +--------+ | +-----------+
| producer1 |---+ | slave | +---| producer2 |
+-----------+ | +--------+ | +-----------+
| |
+-------------+ | | +-------------+
| subscriber1 |-+ +-| subscriber2 |
+-------------+ +-------------+
We have a master/slave broker pair in the middle and two embedded brokers
called embedded-client1 and embedded-client2 that connect to it with
"static:failover://(tcp://localhost:<masterPort>,tcp://localhost:<slavePort>)?randomize=false&maxReconnectAttempts=0
Then we're simulating two clients, one uses the connection URL
"failover://(tcp://localhost:<masterPort>",tcp://localhost:<slavePort>,vm://embedded-
client1)?randomize=false" for sending and subscribing,
the second uses the connection URL
"failover://(tcp://localhost:<masterPort>",tcp://localhost:<slavePort>,vm://embedded-
client2)?randomize=false" for sending and subscribing.
Both clients register a durable subscriber and after both have registered (and
we gave the subscriptions some time to propagate) both clients send a message.
After waiting again some time so the message is delivered we kill the master
and every client sends another message.
At this point I expect the producers to notice that they can neither connect
to the master, nor the slave (because it hasn't had time to start yet) and
falls back to delivering the message to their embedded brokers.
Also I expect the subscribers to notice their connection is gone and subscribe
at their embedded brokers.
So given enough time, both clients should have received the first two messages
and the second one they sent.
Then the slave starts up (in our test we manually start it at this point) and
given enough time, both embedded brokers should reestablish their network
bridge, forward the message that was delivered to them to the central broker
and this should forward the messages to the other embedded broker with it's
subscriber.
So both subscribers should have received all four messages. (In different
order, but that's ok.)
What actually happens is: The test works. once. But when I run it 20 times, it
looks like this:
- The first time always works.
- A statistical average of 6 times it succeeds.
- A statistical average of 4 times I get a JMSException that a client id is
already registered.
- A statistical average of 2 times the clients don't get their own messages
after the central broker died.
- A statistical average of 8 times the clients already received all 4 messages
before the central broker was restarted.
- Even though all brokers have been told to use a temporary directory to store
their data that's wiped after every test run (junit TemporaryFolder rule),
directories "activemq-data/<name-of-embedded-broker1>", "activemq-data/<name-
of-embedded-broker2>" are created in the current directory.
The test case already stops all 4 brokers that are started after the test with
stop() and waitUntilStopped().
I have the impression that sometimes the connection does not go to the
embedded broker that I manually created but a new embedded broker with default
settings is created instead. But I have no clue why this happens sometimes.
Maybe the embedded instances from the last test run aren't disposed properly
yet. But then again I have no clue how to clean the state between test runs
properly.
At the moment the test starts with a 4 second delay, without it the number of
JMSExceptions rises dramatically.
I'll attach a small maven project with this test case, as well as a log file
of a test run where all 4 test outcomes happened.
Any help would be greatly appreciated.
Thanks
Joachim
maven-test-project.zip (9K) <http://activemq.2283324.n4.nabble.com/attachment/4714185/0/maven-test-project.zip>
tests.log.zip (904K) <http://activemq.2283324.n4.nabble.com/attachment/4714185/1/tests.log.zip>
--
View this message in context: http://activemq.2283324.n4.nabble.com/problems-getting-test-results-deterministic-tp4714185.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: problems getting test results deterministic
Posted by jnicolay <j....@topdesk.com>.
Hi
Yes, this shifts the probabilities somewhat, but still doesn't make it work
all the time. In the exact current setup plus ?create=false I got 3 times
missed messages and 17 successes.
But I still have a Thread.sleep at the beginning of every test and a
System.gc which should both not be necessary. And if I remove these two I
get 1 missing message, one success (the first run) and 18 execptions
(InvalidClientIDException, client already connected).
With at least one second delay before every test I get 4 missing messages
(interestingly not always the same) and 16 successes.
--
View this message in context: http://activemq.2283324.n4.nabble.com/problems-getting-test-results-deterministic-tp4714185p4714216.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: problems getting test results deterministic
Posted by Quinn Stevenson <qu...@pronoia-solutions.com>.
Have you tried explicitly disabling the automatic creation of embedded brokers (i.e. add create=false option to vm://…. )
> On Jul 20, 2016, at 9:15 AM, jnicolay <j....@topdesk.com> wrote:
>
> Hi all
>
> We ran into some strange behavior in our application and I tried to reproduce
> it in a unit test.
>
> Our setup looks somewhat like this:
> +-----------+ +--------+ +-----------+
> | embedded1 | <---> | master | <--> | embedded2 |
> +-----------+ +--------+ +-----------+
> \------ ----/ \--- ------/
> \/ \/
> +-----------+ | +--------+ | +-----------+
> | producer1 |---+ | slave | +---| producer2 |
> +-----------+ | +--------+ | +-----------+
> | |
> +-------------+ | | +-------------+
> | subscriber1 |-+ +-| subscriber2 |
> +-------------+ +-------------+
>
>
> We have a master/slave broker pair in the middle and two embedded brokers
> called embedded-client1 and embedded-client2 that connect to it with
> "static:failover://(tcp://localhost:<masterPort>,tcp://localhost:<slavePort>)?randomize=false&maxReconnectAttempts=0
>
> Then we're simulating two clients, one uses the connection URL
> "failover://(tcp://localhost:<masterPort>",tcp://localhost:<slavePort>,vm://embedded-
> client1)?randomize=false" for sending and subscribing,
> the second uses the connection URL
> "failover://(tcp://localhost:<masterPort>",tcp://localhost:<slavePort>,vm://embedded-
> client2)?randomize=false" for sending and subscribing.
>
> Both clients register a durable subscriber and after both have registered (and
> we gave the subscriptions some time to propagate) both clients send a message.
>
> After waiting again some time so the message is delivered we kill the master
> and every client sends another message.
> At this point I expect the producers to notice that they can neither connect
> to the master, nor the slave (because it hasn't had time to start yet) and
> falls back to delivering the message to their embedded brokers.
> Also I expect the subscribers to notice their connection is gone and subscribe
> at their embedded brokers.
>
> So given enough time, both clients should have received the first two messages
> and the second one they sent.
>
> Then the slave starts up (in our test we manually start it at this point) and
> given enough time, both embedded brokers should reestablish their network
> bridge, forward the message that was delivered to them to the central broker
> and this should forward the messages to the other embedded broker with it's
> subscriber.
>
> So both subscribers should have received all four messages. (In different
> order, but that's ok.)
>
> What actually happens is: The test works. once. But when I run it 20 times, it
> looks like this:
> - The first time always works.
> - A statistical average of 6 times it succeeds.
> - A statistical average of 4 times I get a JMSException that a client id is
> already registered.
> - A statistical average of 2 times the clients don't get their own messages
> after the central broker died.
> - A statistical average of 8 times the clients already received all 4 messages
> before the central broker was restarted.
> - Even though all brokers have been told to use a temporary directory to store
> their data that's wiped after every test run (junit TemporaryFolder rule),
> directories "activemq-data/<name-of-embedded-broker1>", "activemq-data/<name-
> of-embedded-broker2>" are created in the current directory.
>
> The test case already stops all 4 brokers that are started after the test with
> stop() and waitUntilStopped().
>
>
> I have the impression that sometimes the connection does not go to the
> embedded broker that I manually created but a new embedded broker with default
> settings is created instead. But I have no clue why this happens sometimes.
> Maybe the embedded instances from the last test run aren't disposed properly
> yet. But then again I have no clue how to clean the state between test runs
> properly.
>
> At the moment the test starts with a 4 second delay, without it the number of
> JMSExceptions rises dramatically.
>
>
> I'll attach a small maven project with this test case, as well as a log file
> of a test run where all 4 test outcomes happened.
>
>
> Any help would be greatly appreciated.
>
> Thanks
> Joachim
>
>
> maven-test-project.zip (9K) <http://activemq.2283324.n4.nabble.com/attachment/4714185/0/maven-test-project.zip>
> tests.log.zip (904K) <http://activemq.2283324.n4.nabble.com/attachment/4714185/1/tests.log.zip>
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/problems-getting-test-results-deterministic-tp4714185.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.