You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomee.apache.org by almos <al...@ukr.net> on 2012/08/11 16:03:30 UTC

Strange MDB invocation behavior

Hello,

I am facing a strange behavior with MDB's that are listening on topics.
After long application inactive time (in other words - system is not used
overnight (3-4 hours)) first JMS message send to the topic NEVER reaches
destination (I have couple of MDBs listening on the topic and neither of
them gets invoked). I.e MDB isn't invoked (according to the verbose logs).
All next subsequent send operations works fine and all messages reach
destinations.
According to our observations this doesn't happen with JMS Queues which uses
the same infrastructure code referenced below.

What might be a problem? What we are doing incorrectly?

Here is a code I use to send messages to the topic (without spaces, looks
like forum engine strips some URLs)
http:// pastebin.com /kZ5W0kXS

So having this code to send messages to topic/queue I use following
approach:

1. inject sender into ejb bean:
@EJB private IISMClient ismClient;
2. invoke send function
ismClient.post(notification);

Here is my tomee.xml (without spaces)
http:// pastebin.com /kcdBcgEw

Here is how we use @MessageDriven annotation. And as I mentioned after long
user inactivity first message sent to the topic ismTopic never gets
delivered to the listeners:

@MessageDriven(activationConfig = {
		@ActivationConfigProperty(
				propertyName = "destinationType",
				propertyValue = "javax.jms.Topic"),
        @ActivationConfigProperty(
        		propertyName = "destination",
        		propertyValue = "ismTopic")
		})
public class TestListener extends AGenericListener implements
MessageListener {
@Override
	public void onMessage(Message msg)
	{
		TextMessage tm = (TextMessage) msg;
		try {
			String message = tm.getText();
			logger.info("TestListener [message]: " + message);
			setRawPacket(message);
			process();
		} catch (JMSException e) {
			e.printStackTrace();
		}
	}
}

Sender and listeners resides currently under the same TomEE 1.0 instance on
the same machine.

Could you please check what I might be doing wrong?

Thanks,
Alex



--
View this message in context: http://openejb.979440.n4.nabble.com/Strange-MDB-invocation-behavior-tp4656801.html
Sent from the OpenEJB User mailing list archive at Nabble.com.

Re: Strange MDB invocation behavior

Posted by almos <al...@ukr.net>.
After a lot of investigation I've found a problem here on forum. I was using
TomEE v1.0 with ActiveMQ client jar v5.5.1 which has some bug or whatever.
Everything works smoothly with latest Tomee v1.1 build.
There are no exceptions, however I am periodically checking logs for any
kind of related problems. I'll let you know if any.
Thanks for the support.



--
View this message in context: http://openejb.979440.n4.nabble.com/Strange-MDB-invocation-behavior-tp4656801p4657321.html
Sent from the OpenEJB User mailing list archive at Nabble.com.

Re: Strange MDB invocation behavior

Posted by Romain Manni-Bucau <rm...@gmail.com>.
I think you need one per tx. But dont worry real connections are normally
pooled for perf.
Le 6 sept. 2012 09:30, "almos" <al...@ukr.net> a écrit :

> I have added more logging and found that container occasionally issues
> following Exception when it sends JMS message.
> What the problem might be? I am creating connection in @PostConstruct
> method
> and then reusing it when posting messages (Session/Producers are created
> per
> send operations and not shared). Posting might occur from different
> threads.
> Should I create connection per message send operation or sharing Connection
> instance is fine?
>
> javax.jms.JMSException: The resource is allready being used in transaction
> context.
>         at
>
> org.apache.activemq.ra.ManagedTransactionContext.setUseSharedTxContext(ManagedTransactionContext.java:47)
>         at
>
> org.apache.activemq.ra.ManagedSessionProxy.setUseSharedTxContext(ManagedSessionProxy.java:67)
>         at
>
> org.apache.activemq.ra.ManagedConnectionProxy.createSessionProxy(ManagedConnectionProxy.java:122)
>         at
>
> org.apache.activemq.ra.ManagedConnectionProxy.createQueueSession(ManagedConnectionProxy.java:150)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at
>
> org.apache.openejb.resource.AutoConnectionTracker$ConnectionInvocationHandler.invoke(AutoConnectionTracker.java:125)
>         at $Proxy75.createQueueSession(Unknown Source)
>         at
>
> com.bmm.dao.jms.AQueueMessageSender.createSession(AQueueMessageSender.java:23)
>         at com.bmm.dao.jms.AMessageSender.post(AMessageSender.java:112)
>         at
> com.bmm.dao.jms.SynchronizerClient.post(SynchronizerClient.java:52)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at
>
> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:181)
>         at
>
> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:163)
>         at
>
> org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:176)
>         at
>
> org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:95)
>         at sun.reflect.GeneratedMethodAccessor118.invoke(Unknown Source)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at
>
> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:181)
>         at
>
> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:163)
>         at
> org.apache.openejb.cdi.CdiInterceptor.invoke(CdiInterceptor.java:129)
>         at
> org.apache.openejb.cdi.CdiInterceptor.access$000(CdiInterceptor.java:45)
>         at
> org.apache.openejb.cdi.CdiInterceptor$1.call(CdiInterceptor.java:66)
>         at
> org.apache.openejb.cdi.CdiInterceptor.aroundInvoke(CdiInterceptor.java:72)
>         at sun.reflect.GeneratedMethodAccessor117.invoke(Unknown Source)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at
>
> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:181)
>         at
>
> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:163)
>         at
>
> org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:138)
>         at
>
> org.apache.openejb.core.stateless.StatelessContainer._invoke(StatelessContainer.java:226)
>         at
>
> org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:178)
>         at
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler$AsynchronousCall.call(EjbObjectProxyHandler.java:296)
>         at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
>
>
>
>
> --
> View this message in context:
> http://openejb.979440.n4.nabble.com/Strange-MDB-invocation-behavior-tp4656801p4657302.html
> Sent from the OpenEJB User mailing list archive at Nabble.com.
>

Re: Strange MDB invocation behavior

Posted by almos <al...@ukr.net>.
I have added more logging and found that container occasionally issues
following Exception when it sends JMS message.
What the problem might be? I am creating connection in @PostConstruct method
and then reusing it when posting messages (Session/Producers are created per
send operations and not shared). Posting might occur from different threads.
Should I create connection per message send operation or sharing Connection
instance is fine?

javax.jms.JMSException: The resource is allready being used in transaction
context.
	at
org.apache.activemq.ra.ManagedTransactionContext.setUseSharedTxContext(ManagedTransactionContext.java:47)
	at
org.apache.activemq.ra.ManagedSessionProxy.setUseSharedTxContext(ManagedSessionProxy.java:67)
	at
org.apache.activemq.ra.ManagedConnectionProxy.createSessionProxy(ManagedConnectionProxy.java:122)
	at
org.apache.activemq.ra.ManagedConnectionProxy.createQueueSession(ManagedConnectionProxy.java:150)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.openejb.resource.AutoConnectionTracker$ConnectionInvocationHandler.invoke(AutoConnectionTracker.java:125)
	at $Proxy75.createQueueSession(Unknown Source)
	at
com.bmm.dao.jms.AQueueMessageSender.createSession(AQueueMessageSender.java:23)
	at com.bmm.dao.jms.AMessageSender.post(AMessageSender.java:112)
	at com.bmm.dao.jms.SynchronizerClient.post(SynchronizerClient.java:52)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:181)
	at
org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:163)
	at
org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:176)
	at
org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:95)
	at sun.reflect.GeneratedMethodAccessor118.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:181)
	at
org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:163)
	at org.apache.openejb.cdi.CdiInterceptor.invoke(CdiInterceptor.java:129)
	at org.apache.openejb.cdi.CdiInterceptor.access$000(CdiInterceptor.java:45)
	at org.apache.openejb.cdi.CdiInterceptor$1.call(CdiInterceptor.java:66)
	at
org.apache.openejb.cdi.CdiInterceptor.aroundInvoke(CdiInterceptor.java:72)
	at sun.reflect.GeneratedMethodAccessor117.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at
org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:181)
	at
org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:163)
	at
org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:138)
	at
org.apache.openejb.core.stateless.StatelessContainer._invoke(StatelessContainer.java:226)
	at
org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:178)
	at
org.apache.openejb.core.ivm.EjbObjectProxyHandler$AsynchronousCall.call(EjbObjectProxyHandler.java:296)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)




--
View this message in context: http://openejb.979440.n4.nabble.com/Strange-MDB-invocation-behavior-tp4656801p4657302.html
Sent from the OpenEJB User mailing list archive at Nabble.com.

Re: Strange MDB invocation behavior

Posted by Romain Manni-Bucau <rm...@gmail.com>.
i have not it in mind but i think AMQ has a kind of check interval parameter

- Romain


2012/8/11 almos <al...@ukr.net>

> We thought of network issue as well but it also reproduces in the case when
> TomEE and ActiveMQ are on the same host.
> For us it also seems like a kind of hibernation, when 1-st message awakes
> some hibernated component and then it becomes alive again but first message
> gets lost.
>
>
>
>
> --
> View this message in context:
> http://openejb.979440.n4.nabble.com/Strange-MDB-invocation-behavior-tp4656801p4656805.html
> Sent from the OpenEJB User mailing list archive at Nabble.com.
>

Re: Strange MDB invocation behavior

Posted by almos <al...@ukr.net>.
We thought of network issue as well but it also reproduces in the case when
TomEE and ActiveMQ are on the same host.
For us it also seems like a kind of hibernation, when 1-st message awakes
some hibernated component and then it becomes alive again but first message
gets lost.




--
View this message in context: http://openejb.979440.n4.nabble.com/Strange-MDB-invocation-behavior-tp4656801p4656805.html
Sent from the OpenEJB User mailing list archive at Nabble.com.

Re: Strange MDB invocation behavior

Posted by Romain Manni-Bucau <rm...@gmail.com>.
yep, check AMQ looks like a kind of hibernation

can be a network issue too, no? then it fails and recover but you lost one
message

- Romain


2012/8/11 almos <al...@ukr.net>

> Not exactly.
>
> System is running all the time. When users are doing some particular
> actions
> we use to send JMS messages  and everything works fine in 99.9% of cases.
>
> After off hours (absolutely no activity at all for about 4-6 hours from the
> users) where the are no requests and hence JMS invocations, first JMS
> message delivery to the topic doesn't occur.
>
> In other words after about of 4-6 hours of inactivity, first user comes to
> the system, does some action that triggers JMS message send operation and
> that message doesn't get delivered to the MDB listeners.
> All subsequent messages send via the system successfully delivered without
> any other efforts done to the application (no shutdown/re-deploy).
> In logs there are no exceptions and any other suspicious things. So the
> only
> one message is lost after off hours (inactivity) and then system works as
> expected till next off hours period after which we face the same 1-st
> message loss problem.
>
> I'll check ability to send via broker managemet console as well, thanks for
> the suggestion.
>
> Regards,
> Alex
>
>
>
> --
> View this message in context:
> http://openejb.979440.n4.nabble.com/Strange-MDB-invocation-behavior-tp4656801p4656803.html
> Sent from the OpenEJB User mailing list archive at Nabble.com.
>

Re: Strange MDB invocation behavior

Posted by almos <al...@ukr.net>.
Not exactly. 

System is running all the time. When users are doing some particular actions
we use to send JMS messages  and everything works fine in 99.9% of cases.
 
After off hours (absolutely no activity at all for about 4-6 hours from the
users) where the are no requests and hence JMS invocations, first JMS
message delivery to the topic doesn't occur.

In other words after about of 4-6 hours of inactivity, first user comes to
the system, does some action that triggers JMS message send operation and
that message doesn't get delivered to the MDB listeners. 
All subsequent messages send via the system successfully delivered without
any other efforts done to the application (no shutdown/re-deploy).
In logs there are no exceptions and any other suspicious things. So the only
one message is lost after off hours (inactivity) and then system works as
expected till next off hours period after which we face the same 1-st
message loss problem.

I'll check ability to send via broker managemet console as well, thanks for
the suggestion.

Regards,
Alex



--
View this message in context: http://openejb.979440.n4.nabble.com/Strange-MDB-invocation-behavior-tp4656801p4656803.html
Sent from the OpenEJB User mailing list archive at Nabble.com.

Re: Strange MDB invocation behavior

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi,

it means it works first then stop working?

can you check you can still send message to AMQ broker (using AMQ
directly), if not please check your AMQ config.

- Romain


2012/8/11 almos <al...@ukr.net>

> Hello,
>
> I am facing a strange behavior with MDB's that are listening on topics.
> After long application inactive time (in other words - system is not used
> overnight (3-4 hours)) first JMS message send to the topic NEVER reaches
> destination (I have couple of MDBs listening on the topic and neither of
> them gets invoked). I.e MDB isn't invoked (according to the verbose logs).
> All next subsequent send operations works fine and all messages reach
> destinations.
> According to our observations this doesn't happen with JMS Queues which
> uses
> the same infrastructure code referenced below.
>
> What might be a problem? What we are doing incorrectly?
>
> Here is a code I use to send messages to the topic (without spaces, looks
> like forum engine strips some URLs)
> http:// pastebin.com /kZ5W0kXS
>
> So having this code to send messages to topic/queue I use following
> approach:
>
> 1. inject sender into ejb bean:
> @EJB private IISMClient ismClient;
> 2. invoke send function
> ismClient.post(notification);
>
> Here is my tomee.xml (without spaces)
> http:// pastebin.com /kcdBcgEw
>
> Here is how we use @MessageDriven annotation. And as I mentioned after long
> user inactivity first message sent to the topic ismTopic never gets
> delivered to the listeners:
>
> @MessageDriven(activationConfig = {
>                 @ActivationConfigProperty(
>                                 propertyName = "destinationType",
>                                 propertyValue = "javax.jms.Topic"),
>         @ActivationConfigProperty(
>                         propertyName = "destination",
>                         propertyValue = "ismTopic")
>                 })
> public class TestListener extends AGenericListener implements
> MessageListener {
> @Override
>         public void onMessage(Message msg)
>         {
>                 TextMessage tm = (TextMessage) msg;
>                 try {
>                         String message = tm.getText();
>                         logger.info("TestListener [message]: " + message);
>                         setRawPacket(message);
>                         process();
>                 } catch (JMSException e) {
>                         e.printStackTrace();
>                 }
>         }
> }
>
> Sender and listeners resides currently under the same TomEE 1.0 instance on
> the same machine.
>
> Could you please check what I might be doing wrong?
>
> Thanks,
> Alex
>
>
>
> --
> View this message in context:
> http://openejb.979440.n4.nabble.com/Strange-MDB-invocation-behavior-tp4656801.html
> Sent from the OpenEJB User mailing list archive at Nabble.com.
>