You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomee.apache.org by Robert Bray <ro...@3ds.com> on 2015/08/17 20:50:00 UTC
java.lang.IllegalStateException: The last message delivery must be
completed wit h an afterDeliver before another message can be delivered
I have this error when I bombard my app with lots of requests which generates
many threads concurrently trying send some JMS messages.
SEVERE: error dispatching message:
java.lang.IllegalStateException: The last message delivery must be completed
wit
h an afterDeliver before another message can be delivered
at
org.apache.openejb.core.mdb.EndpointHandler.deliverMessage(EndpointHa
ndler.java:221)
at
org.apache.openejb.core.mdb.EndpointHandler.invoke(EndpointHandler.ja
va:169)
at my.app.class$$LocalBeanProxy.onMessage(my/app/class.java)
at
org.apache.activemq.ra.MessageEndpointProxy$MessageEndpointAlive.onMe
ssage(MessageEndpointProxy.java:123)
at
org.apache.activemq.ra.MessageEndpointProxy.onMessage(MessageEndpoint
Proxy.java:64)
at org.apache.activemq.ActiveMQSession.run(ActiveMQSession.java:994)
at
org.apache.activemq.ra.ServerSessionImpl.run(ServerSessionImpl.java:1
69)
at
org.apache.geronimo.connector.work.WorkerContext.run(WorkerContext.ja
va:366)
at
org.apache.geronimo.connector.work.pool.NamedRunnable.run(NamedRunnab
le.java:32)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)
I may be looking at things wrong, but I notice that the state is a static
member of EndpointHandler and I am clearly calling this code from several
threads. Is EndpointHandler thread safe?
--
View this message in context: http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921.html
Sent from the TomEE Users mailing list archive at Nabble.com.
Re: java.lang.IllegalStateException: The last message delivery must
be completed wit h an afterDeliver before another message can be delivered
Posted by Romain Manni-Bucau <rm...@gmail.com>.
I'd be curious if you can reproduce it in a sample you can share, also
activating debug logs on org.apache.activemq can help - but take care it
can be very verbose.
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber
<http://www.tomitribe.com>
2015-08-21 13:22 GMT-07:00 Robert Bray <ro...@3ds.com>:
> Romain,
>
> We have 2 MBDs each configured for 100 sessions, so 2*100?
>
> how should we configure it?
>
>
> Romain Manni-Bucau wrote
> > Hi Robert,
> >
> > any reason to use PoolMaxSize=200 and not 100?
> >
> >
> > Romain Manni-Bucau
> > @rmannibucau <https://twitter.com/rmannibucau> | Blog
> > <http://rmannibucau.wordpress.com> | Github
> > <https://github.com/rmannibucau> |
> > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber
> > <http://www.tomitribe.com>
> >
> > 2015-08-21 11:53 GMT-07:00 Robert Bray <
>
> > robert.bray@
>
> > >:
> >
> >> Thanks Romain,
> >>
> >> In the default case, I have the following config
> >>
> > <Resource id="xxxJmsResourceAdapter" type="ActiveMQResourceAdapter">
> >> BrokerXmlConfig =
> >> broker:(nio://0.0.0.0:61616?daemon=true)?persistent=true
> >> DataSource =
> >> threadPoolSize 100
> >>
> > </Resource>
> >>
> >>
> > <Resource id="xxxx/jms/connectionfactory"
> >>
> > type="javax.jms.ConnectionFactory">
> >> ResourceAdapter = xxJmsResourceAdapter
> >> TransactionSupport xa
> >> PoolMaxSize 200
> >>
> > </Resource>
> >>
> >> and
> >>
> > <activation-config-property>
> >>
> >>
> >>
> > <activation-config-property-name>
> > maxSessions
> > </activation-config-property-name>
> >>
> >>
> > <activation-config-property-value>
> > 100
> > </activation-config-property-value>
> >>
> > </activation-config-property>
> >>
> >> With this config I am able to reproduce this error. It appears that the
> >> session are re-used because we reach maxSessions and when that happens
> we
> >> get this error.
> >>
> >>
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921p4675956.html
> >> Sent from the TomEE Users mailing list archive at Nabble.com.
> >>
>
>
>
>
>
> --
> View this message in context:
> http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921p4675958.html
> Sent from the TomEE Users mailing list archive at Nabble.com.
>
Re: java.lang.IllegalStateException: The last message delivery must
be completed wit h an afterDeliver before another message can be delivered
Posted by Robert Bray <ro...@3ds.com>.
Romain,
We have 2 MBDs each configured for 100 sessions, so 2*100?
how should we configure it?
Romain Manni-Bucau wrote
> Hi Robert,
>
> any reason to use PoolMaxSize=200 and not 100?
>
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <http://rmannibucau.wordpress.com> | Github
> <https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber
> <http://www.tomitribe.com>
>
> 2015-08-21 11:53 GMT-07:00 Robert Bray <
> robert.bray@
> >:
>
>> Thanks Romain,
>>
>> In the default case, I have the following config
>>
> <Resource id="xxxJmsResourceAdapter" type="ActiveMQResourceAdapter">
>> BrokerXmlConfig =
>> broker:(nio://0.0.0.0:61616?daemon=true)?persistent=true
>> DataSource =
>> threadPoolSize 100
>>
> </Resource>
>>
>>
> <Resource id="xxxx/jms/connectionfactory"
>>
> type="javax.jms.ConnectionFactory">
>> ResourceAdapter = xxJmsResourceAdapter
>> TransactionSupport xa
>> PoolMaxSize 200
>>
> </Resource>
>>
>> and
>>
> <activation-config-property>
>>
>>
>>
> <activation-config-property-name>
> maxSessions
> </activation-config-property-name>
>>
>>
> <activation-config-property-value>
> 100
> </activation-config-property-value>
>>
> </activation-config-property>
>>
>> With this config I am able to reproduce this error. It appears that the
>> session are re-used because we reach maxSessions and when that happens we
>> get this error.
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921p4675956.html
>> Sent from the TomEE Users mailing list archive at Nabble.com.
>>
--
View this message in context: http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921p4675958.html
Sent from the TomEE Users mailing list archive at Nabble.com.
Re: java.lang.IllegalStateException: The last message delivery must
be completed wit h an afterDeliver before another message can be delivered
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi Robert,
any reason to use PoolMaxSize=200 and not 100?
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber
<http://www.tomitribe.com>
2015-08-21 11:53 GMT-07:00 Robert Bray <ro...@3ds.com>:
> Thanks Romain,
>
> In the default case, I have the following config
> <Resource id="xxxJmsResourceAdapter" type="ActiveMQResourceAdapter">
> BrokerXmlConfig =
> broker:(nio://0.0.0.0:61616?daemon=true)?persistent=true
> DataSource =
> threadPoolSize 100
> </Resource>
>
> <Resource id="xxxx/jms/connectionfactory"
> type="javax.jms.ConnectionFactory">
> ResourceAdapter = xxJmsResourceAdapter
> TransactionSupport xa
> PoolMaxSize 200
> </Resource>
>
> and
> <activation-config-property>
>
>
> <activation-config-property-name>maxSessions</activation-config-property-name>
>
> <activation-config-property-value>100</activation-config-property-value>
> </activation-config-property>
>
> With this config I am able to reproduce this error. It appears that the
> session are re-used because we reach maxSessions and when that happens we
> get this error.
>
>
>
>
>
> --
> View this message in context:
> http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921p4675956.html
> Sent from the TomEE Users mailing list archive at Nabble.com.
>
Re: java.lang.IllegalStateException: The last message delivery must
be completed wit h an afterDeliver before another message can be delivered
Posted by Robert Bray <ro...@3ds.com>.
Thanks Romain,
In the default case, I have the following config
<Resource id="xxxJmsResourceAdapter" type="ActiveMQResourceAdapter">
BrokerXmlConfig =
broker:(nio://0.0.0.0:61616?daemon=true)?persistent=true
DataSource =
threadPoolSize 100
</Resource>
<Resource id="xxxx/jms/connectionfactory"
type="javax.jms.ConnectionFactory">
ResourceAdapter = xxJmsResourceAdapter
TransactionSupport xa
PoolMaxSize 200
</Resource>
and
<activation-config-property>
<activation-config-property-name>maxSessions</activation-config-property-name>
<activation-config-property-value>100</activation-config-property-value>
</activation-config-property>
With this config I am able to reproduce this error. It appears that the
session are re-used because we reach maxSessions and when that happens we
get this error.
--
View this message in context: http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921p4675956.html
Sent from the TomEE Users mailing list archive at Nabble.com.
Re: java.lang.IllegalStateException: The last message delivery must
be completed wit h an afterDeliver before another message can be delivered
Posted by Romain Manni-Bucau <rm...@gmail.com>.
the defaut JMS thread pool size is 30 - should be logged - but if you have
only 10 sessions you can get this kind of issue, do you care adjusting one
of these parameter to the other please?
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber
<http://www.tomitribe.com>
2015-08-20 21:52 GMT-07:00 Robert Bray <ro...@3ds.com>:
> The logging hasn't revealed anything terrible interesting, and I have been
> unable to find any exceptions thrown in the processing of the message
> before
> the error, but I have discovered something interesting:
>
> by decreasing and increasing this setting
>
>
> <activation-config-property-name>maxSessions</activation-config-property-name>
>
> <activation-config-property-value>10</activation-config-property-value>
> </activation-config-property>
> I can make the error occur more and less frequently.
>
> I don't know if this exception is an indication that the concurrency is too
> low and we need to catch the exception and retry the message delivery or if
> this is unexpected behavior.
>
>
>
> --
> View this message in context:
> http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921p4675950.html
> Sent from the TomEE Users mailing list archive at Nabble.com.
>
Re: java.lang.IllegalStateException: The last message delivery must
be completed wit h an afterDeliver before another message can be delivered
Posted by Robert Bray <ro...@3ds.com>.
The logging hasn't revealed anything terrible interesting, and I have been
unable to find any exceptions thrown in the processing of the message before
the error, but I have discovered something interesting:
by decreasing and increasing this setting
<activation-config-property-name>maxSessions</activation-config-property-name>
<activation-config-property-value>10</activation-config-property-value>
</activation-config-property>
I can make the error occur more and less frequently.
I don't know if this exception is an indication that the concurrency is too
low and we need to catch the exception and retry the message delivery or if
this is unexpected behavior.
--
View this message in context: http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921p4675950.html
Sent from the TomEE Users mailing list archive at Nabble.com.
Re: java.lang.IllegalStateException: The last message delivery must
be completed wit h an afterDeliver before another message can be delivered
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Can you turn on debug logging please? in conf/logging.properties - there
should be a commented sample for it but it would look like:
OpenEJB.level=FINEST
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber
<http://www.tomitribe.com>
2015-08-17 12:24 GMT-07:00 Robert Bray <ro...@3ds.com>:
> Thanks Romain,
>
> I knew I was looking at wrong, I blame lack of sleep.
>
> I also was expecting an exception previous to this one.... but none are
> logged.
>
> Any thoughts to pinpoint the exception previous to this one?
>
>
>
> --
> View this message in context:
> http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921p4675923.html
> Sent from the TomEE Users mailing list archive at Nabble.com.
>
Re: java.lang.IllegalStateException: The last message delivery must
be completed wit h an afterDeliver before another message can be delivered
Posted by Robert Bray <ro...@3ds.com>.
Thanks Romain,
I knew I was looking at wrong, I blame lack of sleep.
I also was expecting an exception previous to this one.... but none are
logged.
Any thoughts to pinpoint the exception previous to this one?
--
View this message in context: http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921p4675923.html
Sent from the TomEE Users mailing list archive at Nabble.com.
Re: java.lang.IllegalStateException: The last message delivery must
be completed wit h an afterDeliver before another message can be delivered
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi Robert,
yes it is thread safe, the enum is static but not he value itself.
You should get an exception before this one in the logs. Basically the MDB
throws an exception and then the JMS lifecycle can't be completed properly.
This is what means the message.
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber
<http://www.tomitribe.com>
2015-08-17 11:50 GMT-07:00 Robert Bray <ro...@3ds.com>:
> I have this error when I bombard my app with lots of requests which
> generates
> many threads concurrently trying send some JMS messages.
>
> SEVERE: error dispatching message:
> java.lang.IllegalStateException: The last message delivery must be
> completed
> wit
> h an afterDeliver before another message can be delivered
> at
> org.apache.openejb.core.mdb.EndpointHandler.deliverMessage(EndpointHa
> ndler.java:221)
> at
> org.apache.openejb.core.mdb.EndpointHandler.invoke(EndpointHandler.ja
> va:169)
> at my.app.class$$LocalBeanProxy.onMessage(my/app/class.java)
> at
> org.apache.activemq.ra.MessageEndpointProxy$MessageEndpointAlive.onMe
> ssage(MessageEndpointProxy.java:123)
> at
> org.apache.activemq.ra.MessageEndpointProxy.onMessage(MessageEndpoint
> Proxy.java:64)
> at
> org.apache.activemq.ActiveMQSession.run(ActiveMQSession.java:994)
> at
> org.apache.activemq.ra.ServerSessionImpl.run(ServerSessionImpl.java:1
> 69)
> at
> org.apache.geronimo.connector.work.WorkerContext.run(WorkerContext.ja
> va:366)
> at
> org.apache.geronimo.connector.work.pool.NamedRunnable.run(NamedRunnab
> le.java:32)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
> at java.lang.Thread.run(Unknown Source)
>
> I may be looking at things wrong, but I notice that the state is a static
> member of EndpointHandler and I am clearly calling this code from several
> threads. Is EndpointHandler thread safe?
>
>
>
>
>
>
> --
> View this message in context:
> http://tomee-openejb.979440.n4.nabble.com/java-lang-IllegalStateException-The-last-message-delivery-must-be-completed-wit-h-an-afterDeliver-bed-tp4675921.html
> Sent from the TomEE Users mailing list archive at Nabble.com.
>