Re: MDB / JMS Consumer Throttling?

[David Blevins <da...@gmail.com>]

Shifting this over to dev@ so we can talk code changes.

This code is exactly what you need -- it's what drives our Stateless Container.

 - https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/util/Pool.java

I have to say it's very good.  During my IBM days I worked with a customer for months tuning that code as they had a DCOM based system where each Stateless bean instance held about 100MB of memory and several file handles, so it had to be extremely tight throttling that guaranteed a perfect number of instances and even replaced them in the background if they failed.

There was some work done to leverage this code in the MDB container about 1.5 years ago.  This is a different MDB Container class, so you will need to update your configs:

 - default:  https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/core/mdb/MdbContainer.java
 - pooled:   https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/core/mdb/MdbPoolContainer.java

I didn't review that code, but looking at it now and it should be good to go.  The most critical think to look for as you note is transaction boundaries -- you don't want to start a transaction, then wait for an instance -- you need to wait for an instance then start the transaction when you get one.

Here's what I see for the message delivery flow (it checks out):

 - We grab an instance from Pool here:
     - https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/core/mdb/MdbPoolContainer.java#L272
        - This leads to blocking/throtteling on the pool 
           - https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/core/mdb/MdbInstanceManager.java#L677

 - We now have an instance so we start our transaction via beforeDelivery
     - https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/core/mdb/MdbPoolContainer.java#L275

 - We invoke the MDB

I think what's missing is the documentation and an example.  If you can help with that while you dig, it would be incredibly welcome.


-- 
David Blevins
http://twitter.com/dblevins
http://www.tomitribe.com

> On Feb 26, 2019, at 12:38 PM, exabrial12 <ex...@gmail.com> wrote:
> 
> I know right? It's sort of funny actually. We have one instance of TomEE
> processing 5k+ msg/s for another purpose, but this one is accessing an
> external resource that will error out if we go too fast.
> 
> We can choke resources by limiting the pool and maxSessions, but we need
> something more predictable than that.
> 
> 
> If we add a throttling algorithm to the MDB code, we'll be taking time off
> the JTA timeout, and Thread.sleep() is hardly ever a good idea.
> 
> 
> We could possibly limit it with Camel at the broker level, but we don't want
> to put business logic there.
> 
> 
> If there was a way to inject delay before the JTA timeout started counting,
> that would be ideal. 
> 
> 
> 
> --
> Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html