You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomee.apache.org by Kalyan <ka...@gmail.com> on 2019/10/03 20:16:06 UTC
javax.ejb.ConcurrentAccessTimeoutException: No instances available
in Stateless Session Bean pool. Waited 30 SECONDS
Hello,
I am getting following error in my application.
avax.ejb.ConcurrentAccessTimeoutException: No instances available in
Stateless Session Bean pool. Waited 30 SECONDS
at
org.apache.openejb.core.stateless.StatelessInstanceManager.getInstance(StatelessInstanceManager.java:226)
at
org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
at
org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
at
org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
at
org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
at
org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:349)
My application is heavily used. Looks like not enough instances of beans.
I saw this post to increase the pool size
https://tomee.apache.org/admin/configuration/containers.html
and changed my configuration as
java.naming.factory.initial=org.apache.openejb.core.LocalInitialContextFactory
openejb.deployments.classpath.ear=false
myApp = new://Container?type=STATELESS
myApp.AccessTimeout = 30 seconds
*myApp.MaxSize = 40*
myApp.MinSize = 0
myApp.StrictPooling = true
myApp.MaxAge = 0 hours
myApp.ReplaceAged = true
myApp.ReplaceFlushed = false
myApp.MaxAgeOffset = -1
myApp.IdleTimeout = 0 minutes
myApp.GarbageCollection = false
myApp.SweepInterval = 5 minutes
myApp.CallbackThreads = 5
myApp.CloseTimeout = 5 minutes
myApp.UseOneSchedulerThreadByBean = false
myApp.EvictionThreads = 1
On the server start up I see
INFO - Configuring Service(id=myApp, type=Container, provider-id=Default
Stateless Container)
DEBUG - Containers : 1
DEBUG - Type Container ID
DEBUG - STATELESS myApp
DEBUG - Deployments : 5
DEBUG - Type Deployment ID
/Question i have is /
How would i know if the bean size is increased ? I don't see any log
message for it.
/Secondly I also see /
DEBUG - Using default 'openejb.tempclassloader.skip=none' Possible values
are: none, annotations, enums, all or NONE or ALL
DEBUG - Using default 'AsynchronousPool.Size=5'
DEBUG - Using default 'AsynchronousPool.CorePoolSize=5'
DEBUG - Using default 'AsynchronousPool.MaximumPoolSize=5'
DEBUG - Using default 'AsynchronousPool.QueueSize=5'
DEBUG - Using default 'AsynchronousPool.KeepAliveTime=60 SECONDS'
DEBUG - Using default 'AsynchronousPool.AllowCoreThreadTimeOut=true'
DEBUG - Using default 'AsynchronousPool.QueueType=linked'. Possible values
are: array, linked, priority, synchronous
DEBUG - Using default 'AsynchronousPool.OfferTimeout=30 SECONDS'
DEBUG - Using default 'AsynchronousPool.ShutdownWaitDuration=1 MINUTES'
What's the AsynchronousPool ???
Please help me to sort this issue?
I'm running into this issues in production.
thanks
Kalyan
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
The follow up is appreciated - we hadn't touched upon any of the
asynchronous parts of Kalyan's questions, so any pointers you're able to
give are appreciated. The hanging threads appear to be fixed in 8.0.0.
Jon
On Mon, Oct 14, 2019 at 10:46 AM Richard Monson-Haefel <
monsonhaefel@gmail.com> wrote:
> Please ignore my previous email. I didn't see that Jon had already helped
> with this. Good luck!
>
> On Mon, Oct 14, 2019 at 4:45 AM Richard Monson-Haefel <
> monsonhaefel@gmail.com> wrote:
>
> > Hi Kalyan,
> >
> > Are you using the @Asnycrhous annotation? If so, that's probably why you
> > are seeing messages for the AsynchronousPool. I found this article which
> > helps explain asynchronous EJBs
> >
> > https://www.tomitribe.com/blog/asynchronous-ejbs-on-tomee/
> >
> > It mentions the max pool size is set, as a default of 5. As does this
> > article:
> >
> >
> https://tomee.apache.org/latest/docs/admin/configuration/application.html
> >
> > Have you tried to set that max pool size property to a higher number? If
> > you can determine from logs the average number of invocations per second
> at
> > peak periods you could use that as a guide for setting the async pool
> > size. If that's not possible try a couple different max pool sizes (e.g.
> > 20, 50, 100) and see if that helps. I don't think - although I'm not sure
> > about this - that setting the max pool size higher is going to effect
> > performance much but I'm not sure about that.
> >
> > Richard
> >
> > On Thu, Oct 3, 2019 at 3:16 PM Kalyan <ka...@gmail.com> wrote:
> >
> >> Hello,
> >> I am getting following error in my application.
> >>
> >> avax.ejb.ConcurrentAccessTimeoutException: No instances available in
> >> Stateless Session Bean pool. Waited 30 SECONDS
> >> at
> >>
> >>
> org.apache.openejb.core.stateless.StatelessInstanceManager.getInstance(StatelessInstanceManager.java:226)
> >> at
> >>
> >>
> org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
> >> at
> >>
> >>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
> >> at
> >>
> >>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
> >> at
> >>
> >>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
> >> at
> >>
> >>
> org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:349)
> >>
> >>
> >>
> >> My application is heavily used. Looks like not enough instances of
> beans.
> >>
> >> I saw this post to increase the pool size
> >>
> >> https://tomee.apache.org/admin/configuration/containers.html
> >>
> >>
> >> and changed my configuration as
> >>
> >>
> java.naming.factory.initial=org.apache.openejb.core.LocalInitialContextFactory
> >> openejb.deployments.classpath.ear=false
> >>
> >> myApp = new://Container?type=STATELESS
> >> myApp.AccessTimeout = 30 seconds
> >> *myApp.MaxSize = 40*
> >> myApp.MinSize = 0
> >> myApp.StrictPooling = true
> >> myApp.MaxAge = 0 hours
> >> myApp.ReplaceAged = true
> >> myApp.ReplaceFlushed = false
> >> myApp.MaxAgeOffset = -1
> >> myApp.IdleTimeout = 0 minutes
> >> myApp.GarbageCollection = false
> >> myApp.SweepInterval = 5 minutes
> >> myApp.CallbackThreads = 5
> >> myApp.CloseTimeout = 5 minutes
> >> myApp.UseOneSchedulerThreadByBean = false
> >> myApp.EvictionThreads = 1
> >>
> >>
> >> On the server start up I see
> >>
> >> INFO - Configuring Service(id=myApp, type=Container, provider-id=Default
> >> Stateless Container)
> >>
> >> DEBUG - Containers : 1
> >> DEBUG - Type Container ID
> >> DEBUG - STATELESS myApp
> >> DEBUG - Deployments : 5
> >> DEBUG - Type Deployment ID
> >>
> >> /Question i have is /
> >> How would i know if the bean size is increased ? I don't see any log
> >> message for it.
> >>
> >> /Secondly I also see /
> >>
> >> DEBUG - Using default 'openejb.tempclassloader.skip=none' Possible
> values
> >> are: none, annotations, enums, all or NONE or ALL
> >> DEBUG - Using default 'AsynchronousPool.Size=5'
> >> DEBUG - Using default 'AsynchronousPool.CorePoolSize=5'
> >> DEBUG - Using default 'AsynchronousPool.MaximumPoolSize=5'
> >> DEBUG - Using default 'AsynchronousPool.QueueSize=5'
> >> DEBUG - Using default 'AsynchronousPool.KeepAliveTime=60 SECONDS'
> >> DEBUG - Using default 'AsynchronousPool.AllowCoreThreadTimeOut=true'
> >> DEBUG - Using default 'AsynchronousPool.QueueType=linked'. Possible
> >> values
> >> are: array, linked, priority, synchronous
> >> DEBUG - Using default 'AsynchronousPool.OfferTimeout=30 SECONDS'
> >> DEBUG - Using default 'AsynchronousPool.ShutdownWaitDuration=1 MINUTES'
> >>
> >>
> >> What's the AsynchronousPool ???
> >>
> >>
> >> Please help me to sort this issue?
> >> I'm running into this issues in production.
> >>
> >>
> >>
> >> thanks
> >> Kalyan
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> --
> >> Sent from:
> >> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
> >>
> >
> >
> > --
> > Richard Monson-Haefel
> > https://twitter.com/rmonson
> > https://www.linkedin.com/in/monsonhaefel/
> >
>
>
> --
> Richard Monson-Haefel
> https://twitter.com/rmonson
> https://www.linkedin.com/in/monsonhaefel/
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Richard Monson-Haefel <mo...@gmail.com>.
Please ignore my previous email. I didn't see that Jon had already helped
with this. Good luck!
On Mon, Oct 14, 2019 at 4:45 AM Richard Monson-Haefel <
monsonhaefel@gmail.com> wrote:
> Hi Kalyan,
>
> Are you using the @Asnycrhous annotation? If so, that's probably why you
> are seeing messages for the AsynchronousPool. I found this article which
> helps explain asynchronous EJBs
>
> https://www.tomitribe.com/blog/asynchronous-ejbs-on-tomee/
>
> It mentions the max pool size is set, as a default of 5. As does this
> article:
>
> https://tomee.apache.org/latest/docs/admin/configuration/application.html
>
> Have you tried to set that max pool size property to a higher number? If
> you can determine from logs the average number of invocations per second at
> peak periods you could use that as a guide for setting the async pool
> size. If that's not possible try a couple different max pool sizes (e.g.
> 20, 50, 100) and see if that helps. I don't think - although I'm not sure
> about this - that setting the max pool size higher is going to effect
> performance much but I'm not sure about that.
>
> Richard
>
> On Thu, Oct 3, 2019 at 3:16 PM Kalyan <ka...@gmail.com> wrote:
>
>> Hello,
>> I am getting following error in my application.
>>
>> avax.ejb.ConcurrentAccessTimeoutException: No instances available in
>> Stateless Session Bean pool. Waited 30 SECONDS
>> at
>>
>> org.apache.openejb.core.stateless.StatelessInstanceManager.getInstance(StatelessInstanceManager.java:226)
>> at
>>
>> org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
>> at
>>
>> org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
>> at
>>
>> org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
>> at
>>
>> org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
>> at
>>
>> org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:349)
>>
>>
>>
>> My application is heavily used. Looks like not enough instances of beans.
>>
>> I saw this post to increase the pool size
>>
>> https://tomee.apache.org/admin/configuration/containers.html
>>
>>
>> and changed my configuration as
>>
>> java.naming.factory.initial=org.apache.openejb.core.LocalInitialContextFactory
>> openejb.deployments.classpath.ear=false
>>
>> myApp = new://Container?type=STATELESS
>> myApp.AccessTimeout = 30 seconds
>> *myApp.MaxSize = 40*
>> myApp.MinSize = 0
>> myApp.StrictPooling = true
>> myApp.MaxAge = 0 hours
>> myApp.ReplaceAged = true
>> myApp.ReplaceFlushed = false
>> myApp.MaxAgeOffset = -1
>> myApp.IdleTimeout = 0 minutes
>> myApp.GarbageCollection = false
>> myApp.SweepInterval = 5 minutes
>> myApp.CallbackThreads = 5
>> myApp.CloseTimeout = 5 minutes
>> myApp.UseOneSchedulerThreadByBean = false
>> myApp.EvictionThreads = 1
>>
>>
>> On the server start up I see
>>
>> INFO - Configuring Service(id=myApp, type=Container, provider-id=Default
>> Stateless Container)
>>
>> DEBUG - Containers : 1
>> DEBUG - Type Container ID
>> DEBUG - STATELESS myApp
>> DEBUG - Deployments : 5
>> DEBUG - Type Deployment ID
>>
>> /Question i have is /
>> How would i know if the bean size is increased ? I don't see any log
>> message for it.
>>
>> /Secondly I also see /
>>
>> DEBUG - Using default 'openejb.tempclassloader.skip=none' Possible values
>> are: none, annotations, enums, all or NONE or ALL
>> DEBUG - Using default 'AsynchronousPool.Size=5'
>> DEBUG - Using default 'AsynchronousPool.CorePoolSize=5'
>> DEBUG - Using default 'AsynchronousPool.MaximumPoolSize=5'
>> DEBUG - Using default 'AsynchronousPool.QueueSize=5'
>> DEBUG - Using default 'AsynchronousPool.KeepAliveTime=60 SECONDS'
>> DEBUG - Using default 'AsynchronousPool.AllowCoreThreadTimeOut=true'
>> DEBUG - Using default 'AsynchronousPool.QueueType=linked'. Possible
>> values
>> are: array, linked, priority, synchronous
>> DEBUG - Using default 'AsynchronousPool.OfferTimeout=30 SECONDS'
>> DEBUG - Using default 'AsynchronousPool.ShutdownWaitDuration=1 MINUTES'
>>
>>
>> What's the AsynchronousPool ???
>>
>>
>> Please help me to sort this issue?
>> I'm running into this issues in production.
>>
>>
>>
>> thanks
>> Kalyan
>>
>>
>>
>>
>>
>>
>>
>> --
>> Sent from:
>> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>>
>
>
> --
> Richard Monson-Haefel
> https://twitter.com/rmonson
> https://www.linkedin.com/in/monsonhaefel/
>
--
Richard Monson-Haefel
https://twitter.com/rmonson
https://www.linkedin.com/in/monsonhaefel/
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Richard Monson-Haefel <mo...@gmail.com>.
Hi Kalyan,
Are you using the @Asnycrhous annotation? If so, that's probably why you
are seeing messages for the AsynchronousPool. I found this article which
helps explain asynchronous EJBs
https://www.tomitribe.com/blog/asynchronous-ejbs-on-tomee/
It mentions the max pool size is set, as a default of 5. As does this
article:
https://tomee.apache.org/latest/docs/admin/configuration/application.html
Have you tried to set that max pool size property to a higher number? If
you can determine from logs the average number of invocations per second at
peak periods you could use that as a guide for setting the async pool
size. If that's not possible try a couple different max pool sizes (e.g.
20, 50, 100) and see if that helps. I don't think - although I'm not sure
about this - that setting the max pool size higher is going to effect
performance much but I'm not sure about that.
Richard
On Thu, Oct 3, 2019 at 3:16 PM Kalyan <ka...@gmail.com> wrote:
> Hello,
> I am getting following error in my application.
>
> avax.ejb.ConcurrentAccessTimeoutException: No instances available in
> Stateless Session Bean pool. Waited 30 SECONDS
> at
>
> org.apache.openejb.core.stateless.StatelessInstanceManager.getInstance(StatelessInstanceManager.java:226)
> at
>
> org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
> at
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
> at
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
> at
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
> at
>
> org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:349)
>
>
>
> My application is heavily used. Looks like not enough instances of beans.
>
> I saw this post to increase the pool size
>
> https://tomee.apache.org/admin/configuration/containers.html
>
>
> and changed my configuration as
>
> java.naming.factory.initial=org.apache.openejb.core.LocalInitialContextFactory
> openejb.deployments.classpath.ear=false
>
> myApp = new://Container?type=STATELESS
> myApp.AccessTimeout = 30 seconds
> *myApp.MaxSize = 40*
> myApp.MinSize = 0
> myApp.StrictPooling = true
> myApp.MaxAge = 0 hours
> myApp.ReplaceAged = true
> myApp.ReplaceFlushed = false
> myApp.MaxAgeOffset = -1
> myApp.IdleTimeout = 0 minutes
> myApp.GarbageCollection = false
> myApp.SweepInterval = 5 minutes
> myApp.CallbackThreads = 5
> myApp.CloseTimeout = 5 minutes
> myApp.UseOneSchedulerThreadByBean = false
> myApp.EvictionThreads = 1
>
>
> On the server start up I see
>
> INFO - Configuring Service(id=myApp, type=Container, provider-id=Default
> Stateless Container)
>
> DEBUG - Containers : 1
> DEBUG - Type Container ID
> DEBUG - STATELESS myApp
> DEBUG - Deployments : 5
> DEBUG - Type Deployment ID
>
> /Question i have is /
> How would i know if the bean size is increased ? I don't see any log
> message for it.
>
> /Secondly I also see /
>
> DEBUG - Using default 'openejb.tempclassloader.skip=none' Possible values
> are: none, annotations, enums, all or NONE or ALL
> DEBUG - Using default 'AsynchronousPool.Size=5'
> DEBUG - Using default 'AsynchronousPool.CorePoolSize=5'
> DEBUG - Using default 'AsynchronousPool.MaximumPoolSize=5'
> DEBUG - Using default 'AsynchronousPool.QueueSize=5'
> DEBUG - Using default 'AsynchronousPool.KeepAliveTime=60 SECONDS'
> DEBUG - Using default 'AsynchronousPool.AllowCoreThreadTimeOut=true'
> DEBUG - Using default 'AsynchronousPool.QueueType=linked'. Possible values
> are: array, linked, priority, synchronous
> DEBUG - Using default 'AsynchronousPool.OfferTimeout=30 SECONDS'
> DEBUG - Using default 'AsynchronousPool.ShutdownWaitDuration=1 MINUTES'
>
>
> What's the AsynchronousPool ???
>
>
> Please help me to sort this issue?
> I'm running into this issues in production.
>
>
>
> thanks
> Kalyan
>
>
>
>
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
--
Richard Monson-Haefel
https://twitter.com/rmonson
https://www.linkedin.com/in/monsonhaefel/
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Its 8.0.0-M3
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
The problem with getting single threads here and there is its difficult to
pin down the root cause. With the whole jstack file I can probably give you
some pointers pretty quickly (or I'll just say if not). A very long time
back (in the 1.7.3 era) there was an issue where you can lock a database
pool, but that was actually reproducible under small load with small pools.
Most of these types of issues (that I see, at least), particularly when you
see it in production but not other environments, are bottlenecks
communicating to the database or another external system. One thing to
remember is that your stateless pool will have a max of 40 instances *per
EJB* (not in total). If your database pool is also 40, you'll potentially
see issues there.
The first thread you sent is seeing contention issues trying to get a
database connection (the pool is exhausted, either leaking connections, or
the connections are in use in other threads). The more recent one is
showing contention on the stateless EJB pool (it just hasn't timedout
waiting yet). We can't learn to much from these, other than the fact that
they are waiting. What the other threads are doing will be of interest. I
need the full picture otherwise I'm guessing, which won't really help
either of us.
Jon
On Mon, Oct 7, 2019 at 11:39 PM Kalyan <ka...@gmail.com> wrote:
> Rest of the thread is pretty much as below
>
> "JungoThreadPool-1-110" #220 prio=5 os_prio=0 tid=0x00007fa5fc119000
> nid=0x31ff waiting on condition [0x00007fa56dd75000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x000000068339f010> (a
> java.util.concurrent.SynchronousQueue$TransferStack)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
>
> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)
> at
>
> java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
> at
> java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)
> at
>
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "JungoThreadPool-1-109-T888" #219 prio=5 os_prio=0 tid=0x00007fa5fc117000
> nid=0x31fe waiting on condition [0x00007fa56de74000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x000000068022de00> (a
> java.util.concurrent.Semaphore$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:409)
> at org.apache.openejb.util.Pool.pop(Pool.java:247)
> at org.apache.openejb.util.Pool.pop(Pool.java:228)
> at
>
> org.apache.openejb.core.stateless.StatelessInstanceManager$Data.poolPop(StatelessInstanceManager.java:512)
> at
>
> org.apache.openejb.core.stateless.StatelessInstanceManager.getInstance(StatelessInstanceManager.java:217)
> at
>
> org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
> at
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
> at
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
> at
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
> at
>
> org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:349)
> at com.sun.proxy.$Proxy74.forwardToServiceBean(Unknown Source)
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Rest of the thread is pretty much as below
"JungoThreadPool-1-110" #220 prio=5 os_prio=0 tid=0x00007fa5fc119000
nid=0x31ff waiting on condition [0x00007fa56dd75000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000000068339f010> (a
java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- None
"JungoThreadPool-1-109-T888" #219 prio=5 os_prio=0 tid=0x00007fa5fc117000
nid=0x31fe waiting on condition [0x00007fa56de74000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000000068022de00> (a
java.util.concurrent.Semaphore$NonfairSync)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:409)
at org.apache.openejb.util.Pool.pop(Pool.java:247)
at org.apache.openejb.util.Pool.pop(Pool.java:228)
at
org.apache.openejb.core.stateless.StatelessInstanceManager$Data.poolPop(StatelessInstanceManager.java:512)
at
org.apache.openejb.core.stateless.StatelessInstanceManager.getInstance(StatelessInstanceManager.java:217)
at
org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
at
org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
at
org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
at
org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
at
org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:349)
at com.sun.proxy.$Proxy74.forwardToServiceBean(Unknown Source)
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Thanks a lot for the help Jon!
Regards
Kalyan
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
sure i will try.
Once more thing. Somehow i was not getting logs initially. But when i
changed like below.
It was started logging.
// private static final org.apache.openejb.util.Logger debugLogger =
org.apache.openejb.util.Logger.getInstance(LogCategory.OPENEJB_RESOURCE_JDBC,
BeanContext.class);
private static final org.slf4j.Logger debugLogger =
LoggerFactory.getLogger("Transaction");
Logs
2019-10-09 21:01:30,450 DEBUG [Transaction] (JungoThreadPool-1-194-T784)
getConnection() - active: 23. Called from java.lang.Throwable
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
Setting MaxTotal, InitialSize, MinIdle, and MaxIdle enable you to control
whether and when the pool will grow or shrink -
http://commons.apache.org/proper/commons-dbcp/configuration.html.
I'd avoid the urge to change more parameters if you can as it adds more
variables. It looks like a code change between 8.0.0-M3 and 8.0.0 makes the
difference. I'd be interested to know what change impacts it. If you had
time to do a git bisect, that would be useful to close the loop.
Jon
On Wed, Oct 9, 2019 at 11:56 PM Kalyan <ka...@gmail.com> wrote:
> Hi Jon,
>
> With 8.0.0 version. I don't see any ConcurrentAccessTimeoutException
> happening.
> I ran tests for few mins and works fine.
>
> One more question
>
> I was just going through
> http://commons.apache.org/proper/commons-dbcp/configuration.html
>
> Do you think its require to set MaxIdle for the database connections ?
> does it have to be same as MaxTotal ?
>
> Please advice.
>
> Thanks a lot for the help.
>
> Regards
> Kalyan
>
>
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Hi Jon,
With 8.0.0 version. I don't see any ConcurrentAccessTimeoutException
happening.
I ran tests for few mins and works fine.
One more question
I was just going through
http://commons.apache.org/proper/commons-dbcp/configuration.html
Do you think its require to set MaxIdle for the database connections ?
does it have to be same as MaxTotal ?
Please advice.
Thanks a lot for the help.
Regards
Kalyan
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
Can you give it a shot with the 8.0.0 (final) version?
https://repo1.maven.org/maven2/org/apache/tomee/openejb-core/8.0.0/openejb-core-8.0.0.jar
.
Also, try the patch, but with the debug logging for OpenEJB.resource.jdbc
off.
My change should only be introducing logging, so maybe there is something
else in the mix. It's good news that you can reproduce with a load test.
Thanks
Jon
On Wed, Oct 9, 2019 at 8:11 PM Kalyan <ka...@gmail.com> wrote:
> Hello Jon,
> I just put your code BasicManagedDataSource and tested with some load
> test.
> I don't see any ConcurrentAccessTimeoutException.
>
>
> Below is log
>
> 2019-10-09 18:56:40,815 DEBUG (JungoThreadPool-1-198-T642) close() -
> active:
> 5. Called from java.lang.Throwable
>
>
> But if i put back the old openejb-core-8.0.0-M3.jar. After few runs i see
> ConcurrentAccessTimeoutException.
>
>
> Please let me know your inputs.
>
> thanks
> Kalyan
>
>
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Hello Jon,
I just put your code BasicManagedDataSource and tested with some load test.
I don't see any ConcurrentAccessTimeoutException.
Below is log
2019-10-09 18:56:40,815 DEBUG (JungoThreadPool-1-198-T642) close() - active:
5. Called from java.lang.Throwable
But if i put back the old openejb-core-8.0.0-M3.jar. After few runs i see
ConcurrentAccessTimeoutException.
Please let me know your inputs.
thanks
Kalyan
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Hello Jon,
I just put your code BasicManagedDataSource and tested with some load test.
I don't see any ConcurrentAccessTimeoutException.
Below is log
2019-10-09 18:56:40,815 DEBUG (JungoThreadPool-1-198-T642) close() - active:
5. Called from java.lang.Throwable
But if i put back the old openejb-core-8.0.0-M3.jar. After few runs i see
ConcurrentAccessTimeoutException.
Please let me know your inputs.
thanks
Kalyan
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
The test failures look unrelated, I get the same with master.
To test it out, I'd suggest building OpenEJB with that patch, and trying it
on your own development machine with a database connection pool size of
one. If your connection isn't returned the pool, you'll know about it
quickly :)
Jon
On Wed, 9 Oct 2019, 16:16 Kalyan, <ka...@gmail.com> wrote:
> oh okay.
> thanks for the update
>
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
oh okay.
thanks for the update
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
FYI There's a handful of test failures on my full build here. Not sure if
they are related to my change or not yet.
Jon
On Tue, Oct 8, 2019 at 10:28 PM Kalyan <ka...@gmail.com> wrote:
> Thanks Jon!
> I will try this.
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Thanks Jon!
I will try this.
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
Played around with this a little and opened this PR:
https://github.com/apache/tomee/pull/583. Take a look and see what you
think. I still need to do a full build and make sure I haven't
broken anything here. The theory is that is you turn the logging
on OpenEJB.resource.jdbc up to FINEST, you'll get a bunch of debug output
showing where connections are obtained from the datasource, and where they
are returned with connection.close(). If a transaction completes and the
connection has not been closed, it'll log that out, along with where the
connection was borrowed.
I wouldn't put this in production as it needs review and merging etc, as
well as being very verbose, but if you have a local copy of your
application that you can test with, it might be worth stepping through and
using it to ensure your connections are getting "closed", and returned to
the pool. From what you've indicated so far, it looks like you're leaking
connections (i.e., not calling close() on them).
Jon
On Tue, Oct 8, 2019 at 8:58 PM Kalyan <ka...@gmail.com> wrote:
> yes that's correct
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
yes that's correct
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
I can reproduce your stack trace by deliberately not calling
connection.close().
"main" #1 prio=5 os_prio=31 cpu=1929.67ms elapsed=28.55s
tid=0x00007f8d1c800800 nid=0x1703 waiting on condition [0x0000700000dd1000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.1/Native Method)
- parking to wait for <0x000000070ea81590> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.1
/LockSupport.java:194)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.1
/AbstractQueuedSynchronizer.java:2081)
at
org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:583)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:442)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at
org.apache.commons.dbcp2.managed.ManagedConnection.updateTransactionStatus(ManagedConnection.java:127)
at
org.apache.commons.dbcp2.managed.ManagedConnection.<init>(ManagedConnection.java:60)
at
org.apache.openejb.resource.jdbc.dbcp.BasicManagedDataSource$1$1.<init>(BasicManagedDataSource.java:90)
at
org.apache.openejb.resource.jdbc.dbcp.BasicManagedDataSource$1.getConnection(BasicManagedDataSource.java:90)
at
org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533)
at
org.apache.openejb.resource.jdbc.ManagedDataSourceExhaustionTest$DBWorker.check(ManagedDataSourceExhaustionTest.java:74)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.1/Native
Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.1
/NativeMethodAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.1
/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.1/Method.java:566)
at
org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205)
at
org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186)
at
org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:191)
at
org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:102)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.1/Native
Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.1
/NativeMethodAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.1
/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.1/Method.java:566)
at
org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205)
at
org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186)
at
org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:85)
at
org.apache.openejb.core.singleton.SingletonContainer._invoke(SingletonContainer.java:272)
at
org.apache.openejb.core.singleton.SingletonContainer.invoke(SingletonContainer.java:221)
at
org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
at
org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
at
org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
at
org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:349)
at
org.apache.openejb.resource.jdbc.ManagedDataSourceExhaustionTest$DBWorker$$LocalBeanProxy.check(org/apache/openejb/resource/jdbc/ManagedDataSourceExhaustionTest$DBWorker.java)
at
org.apache.openejb.resource.jdbc.ManagedDataSourceExhaustionTest.test(ManagedDataSourceExhaustionTest.java:84)
Is the EJB making the call out to the database running in a transaction?
Jon
On Tue, Oct 8, 2019 at 7:36 PM Kalyan <ka...@gmail.com> wrote:
> Yes i am able to reproduce this issue.
>
>
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Yes i am able to reproduce this issue.
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
We're probably going to make some small changes quite rapidly to 1) see if
we can reproduce your issue and 2) extract some more information in logs.
If you're willing and able to pick up some snapshots, or patch your
8.0.0-M3 with some PRs and report back, that helps.
Jon
On Tue, Oct 8, 2019 at 6:49 PM Kalyan <ka...@gmail.com> wrote:
> Thanks Jon for looking into it.
> What do you suggest me to do ?
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Thanks Jon for looking into it.
What do you suggest me to do ?
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
At first glance, I'd say it looks like your database connection pool is
exhausted but there's nothing using the database, so either we have a bug
or connections are leaking. Next step will probably be some logging around
that.
Jon
On Tue, 8 Oct 2019, 17:10 Jonathan Gallimore, <jo...@gmail.com>
wrote:
> Got it. Going to be away from my computer for a couple of hours, but I'll
> take a look later today.
>
> Jon
>
> On Tue, Oct 8, 2019 at 4:50 PM Kalyan <ka...@gmail.com> wrote:
>
>> Hi Jon,
>> I sent you whole jstack to your email. Please take a look and let me know
>> if
>> anything can be done.
>>
>> thanks
>> Kalyan
>>
>>
>>
>> --
>> Sent from:
>> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>>
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
Got it. Going to be away from my computer for a couple of hours, but I'll
take a look later today.
Jon
On Tue, Oct 8, 2019 at 4:50 PM Kalyan <ka...@gmail.com> wrote:
> Hi Jon,
> I sent you whole jstack to your email. Please take a look and let me know
> if
> anything can be done.
>
> thanks
> Kalyan
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Hi Jon,
I sent you whole jstack to your email. Please take a look and let me know if
anything can be done.
thanks
Kalyan
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
Are you able to share the whole thing? That's just one thread. If you have
a whole bunch of threads all hitting the database, that sounds like a
potential issue at the database end. If there are no other threads
accessing the database, then its possible there is a leak somewhere.
Jon
On Mon, Oct 7, 2019 at 11:23 PM Kalyan <ka...@gmail.com> wrote:
> Hello Jon,
> Even after increasing the bean pool size. I am getting the exception.
>
> Below is the jstack.
> But I have enough connection configured on the database as well.
>
> Could you please see what's going with this.
>
>
> "JungoThreadPool-1-115-T348" #225 prio=5 os_prio=0 tid=0x00007fa5fc123000
> nid=0x3204 waiting on condition [0x00007fa56d86d000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000006819f6760> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
>
> org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:583)
> at
>
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:442)
> at
>
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
> at
>
> org.apache.commons.dbcp2.managed.ManagedConnection.updateTransactionStatus(ManagedConnection.java:127)
> at
>
> org.apache.commons.dbcp2.managed.ManagedConnection.<init>(ManagedConnection.java:60)
> at
>
> org.apache.openejb.resource.jdbc.dbcp.BasicManagedDataSource$1$1.<init>(BasicManagedDataSource.java:89)
> at
>
> org.apache.openejb.resource.jdbc.dbcp.BasicManagedDataSource$1.getConnection(BasicManagedDataSource.java:89)
> at
>
> org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533)
> at JDBCQueryElement.getConnection(DLJDBCQueryElement.java:3173)
>
> thanks
> Kalyan
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Hello Jon,
Even after increasing the bean pool size. I am getting the exception.
Below is the jstack.
But I have enough connection configured on the database as well.
Could you please see what's going with this.
"JungoThreadPool-1-115-T348" #225 prio=5 os_prio=0 tid=0x00007fa5fc123000
nid=0x3204 waiting on condition [0x00007fa56d86d000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006819f6760> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:583)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:442)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at
org.apache.commons.dbcp2.managed.ManagedConnection.updateTransactionStatus(ManagedConnection.java:127)
at
org.apache.commons.dbcp2.managed.ManagedConnection.<init>(ManagedConnection.java:60)
at
org.apache.openejb.resource.jdbc.dbcp.BasicManagedDataSource$1$1.<init>(BasicManagedDataSource.java:89)
at
org.apache.openejb.resource.jdbc.dbcp.BasicManagedDataSource$1.getConnection(BasicManagedDataSource.java:89)
at
org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533)
at JDBCQueryElement.getConnection(DLJDBCQueryElement.java:3173)
thanks
Kalyan
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
Thanks Jon,
Really appreciate for taking a look at code.
Certainly i will look into the jstack. We had similar issue when this
application was running in jboss too. So first thing we want to do is
increase the pool size. Absolutely i will try to get into the root cause of
this issue.
We don't see this in our lower environments , may be we don't have too much
traffic like production. I will try to do some load test and see if i can
reproduce this issue on dev environment.
Its very simple setup what i did on netty. Once i am out of this issue. I
will send you those details.
Thanks a lot
Regards
Kalyan
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
I've had a quick run through of the code and I can't see a log statement
for that value. We can certainly add it to master, but that might not help
you on 8.0.0-M3. If you're feeling handy with a debugger, put a breakpoint
here:
https://github.com/apache/tomee/blob/tomee-8.0.0-M3/container/openejb-core/src/main/java/org/apache/openejb/core/stateless/StatelessContainerFactory.java#L138
and
inspect the pool field.
Do try and get a jstack. It should just be a matter of running jstack
<pid>. If you're not familiar with the output, you basically get a stack
trace for every thread in the JVM. Check how many threads are trying to
execute your EJB.
When you run into this issue, its always very tempting to just bump up the
pool size. I get asked this question a lot. My personal advice is that
increasing the pool size should be the absolute last thing you do. If you
have 10 threads all executing a business method on your EJB, and they are
all taking longer than 30 seconds, you have an issue and you need to find
the root cause, understand it, and fix it. Bumping it up to 40 will
probably just lead to 40 threads stuck executing your EJB business method
and getting stuck - the problem just ends up getting worse. If you know the
performance measurements, have everything tuned, and need more throughput,
that's the point to look at the pool size.
Also, take a look at the Singleton bean type as opposed to stateless. As
long as you really do not have state in your bean, a singleton
with @Lock(LockType.READ) should work well, and isn't pooled so there's no
limit on the number of threads that can call business methods all at once.
https://github.com/apache/tomee/tree/master/examples/simple-singleton
> our servers are basically using netty. OpenEjb is embedded in it.
This sounds like a very cool setup. If you wanted to contribute that to
community, I'd encourage you to do so. I would certainly be interested in
seeing it, and I'm sure others would be too.
Jon
On Thu, Oct 3, 2019 at 11:21 PM Kalyan <ka...@gmail.com> wrote:
> our servers are basically using netty. OpenEjb is embedded in it. I can try
> to get jstack and look into it.
> Is there anything which is logged while server start up to see how much is
> the max size? I don't have JMX
>
> thanks
> Kalyan
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Kalyan <ka...@gmail.com>.
our servers are basically using netty. OpenEjb is embedded in it. I can try
to get jstack and look into it.
Is there anything which is logged while server start up to see how much is
the max size? I don't have JMX
thanks
Kalyan
--
Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
I dropped your settings into TomEE 8 here, and checked the pool for my EJB
via JMX (openejb.management -> Pool -> openejb -> <empty> -> (file path) ->
(bean) -> Attributes. It showed with a max size of 40. 1 instance pooled,
but I'd expect that to grow to the max size.
I'd suggest getting a jstack and looking for threads
with org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
in the trace, and see if you have something locked or running slowly. I'll
happily look at a jstack if you wish, but it may have some information you
don't want to post publicly (check the content of the jstack before
posting).
Jon
On Thu, Oct 3, 2019 at 9:50 PM Jonathan Gallimore <
jonathan.gallimore@gmail.com> wrote:
> Hi
>
> What version of TomEE are you using?
>
> Jon
>
> On Thu, Oct 3, 2019 at 9:16 PM Kalyan <ka...@gmail.com> wrote:
>
>> Hello,
>> I am getting following error in my application.
>>
>> avax.ejb.ConcurrentAccessTimeoutException: No instances available in
>> Stateless Session Bean pool. Waited 30 SECONDS
>> at
>>
>> org.apache.openejb.core.stateless.StatelessInstanceManager.getInstance(StatelessInstanceManager.java:226)
>> at
>>
>> org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
>> at
>>
>> org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
>> at
>>
>> org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
>> at
>>
>> org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
>> at
>>
>> org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:349)
>>
>>
>>
>> My application is heavily used. Looks like not enough instances of beans.
>>
>> I saw this post to increase the pool size
>>
>> https://tomee.apache.org/admin/configuration/containers.html
>>
>>
>> and changed my configuration as
>>
>> java.naming.factory.initial=org.apache.openejb.core.LocalInitialContextFactory
>> openejb.deployments.classpath.ear=false
>>
>> myApp = new://Container?type=STATELESS
>> myApp.AccessTimeout = 30 seconds
>> *myApp.MaxSize = 40*
>> myApp.MinSize = 0
>> myApp.StrictPooling = true
>> myApp.MaxAge = 0 hours
>> myApp.ReplaceAged = true
>> myApp.ReplaceFlushed = false
>> myApp.MaxAgeOffset = -1
>> myApp.IdleTimeout = 0 minutes
>> myApp.GarbageCollection = false
>> myApp.SweepInterval = 5 minutes
>> myApp.CallbackThreads = 5
>> myApp.CloseTimeout = 5 minutes
>> myApp.UseOneSchedulerThreadByBean = false
>> myApp.EvictionThreads = 1
>>
>>
>> On the server start up I see
>>
>> INFO - Configuring Service(id=myApp, type=Container, provider-id=Default
>> Stateless Container)
>>
>> DEBUG - Containers : 1
>> DEBUG - Type Container ID
>> DEBUG - STATELESS myApp
>> DEBUG - Deployments : 5
>> DEBUG - Type Deployment ID
>>
>> /Question i have is /
>> How would i know if the bean size is increased ? I don't see any log
>> message for it.
>>
>> /Secondly I also see /
>>
>> DEBUG - Using default 'openejb.tempclassloader.skip=none' Possible values
>> are: none, annotations, enums, all or NONE or ALL
>> DEBUG - Using default 'AsynchronousPool.Size=5'
>> DEBUG - Using default 'AsynchronousPool.CorePoolSize=5'
>> DEBUG - Using default 'AsynchronousPool.MaximumPoolSize=5'
>> DEBUG - Using default 'AsynchronousPool.QueueSize=5'
>> DEBUG - Using default 'AsynchronousPool.KeepAliveTime=60 SECONDS'
>> DEBUG - Using default 'AsynchronousPool.AllowCoreThreadTimeOut=true'
>> DEBUG - Using default 'AsynchronousPool.QueueType=linked'. Possible
>> values
>> are: array, linked, priority, synchronous
>> DEBUG - Using default 'AsynchronousPool.OfferTimeout=30 SECONDS'
>> DEBUG - Using default 'AsynchronousPool.ShutdownWaitDuration=1 MINUTES'
>>
>>
>> What's the AsynchronousPool ???
>>
>>
>> Please help me to sort this issue?
>> I'm running into this issues in production.
>>
>>
>>
>> thanks
>> Kalyan
>>
>>
>>
>>
>>
>>
>>
>> --
>> Sent from:
>> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>>
>
Re: javax.ejb.ConcurrentAccessTimeoutException: No instances
available in Stateless Session Bean pool. Waited 30 SECONDS
Posted by Jonathan Gallimore <jo...@gmail.com>.
Hi
What version of TomEE are you using?
Jon
On Thu, Oct 3, 2019 at 9:16 PM Kalyan <ka...@gmail.com> wrote:
> Hello,
> I am getting following error in my application.
>
> avax.ejb.ConcurrentAccessTimeoutException: No instances available in
> Stateless Session Bean pool. Waited 30 SECONDS
> at
>
> org.apache.openejb.core.stateless.StatelessInstanceManager.getInstance(StatelessInstanceManager.java:226)
> at
>
> org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
> at
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
> at
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
> at
>
> org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
> at
>
> org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:349)
>
>
>
> My application is heavily used. Looks like not enough instances of beans.
>
> I saw this post to increase the pool size
>
> https://tomee.apache.org/admin/configuration/containers.html
>
>
> and changed my configuration as
>
> java.naming.factory.initial=org.apache.openejb.core.LocalInitialContextFactory
> openejb.deployments.classpath.ear=false
>
> myApp = new://Container?type=STATELESS
> myApp.AccessTimeout = 30 seconds
> *myApp.MaxSize = 40*
> myApp.MinSize = 0
> myApp.StrictPooling = true
> myApp.MaxAge = 0 hours
> myApp.ReplaceAged = true
> myApp.ReplaceFlushed = false
> myApp.MaxAgeOffset = -1
> myApp.IdleTimeout = 0 minutes
> myApp.GarbageCollection = false
> myApp.SweepInterval = 5 minutes
> myApp.CallbackThreads = 5
> myApp.CloseTimeout = 5 minutes
> myApp.UseOneSchedulerThreadByBean = false
> myApp.EvictionThreads = 1
>
>
> On the server start up I see
>
> INFO - Configuring Service(id=myApp, type=Container, provider-id=Default
> Stateless Container)
>
> DEBUG - Containers : 1
> DEBUG - Type Container ID
> DEBUG - STATELESS myApp
> DEBUG - Deployments : 5
> DEBUG - Type Deployment ID
>
> /Question i have is /
> How would i know if the bean size is increased ? I don't see any log
> message for it.
>
> /Secondly I also see /
>
> DEBUG - Using default 'openejb.tempclassloader.skip=none' Possible values
> are: none, annotations, enums, all or NONE or ALL
> DEBUG - Using default 'AsynchronousPool.Size=5'
> DEBUG - Using default 'AsynchronousPool.CorePoolSize=5'
> DEBUG - Using default 'AsynchronousPool.MaximumPoolSize=5'
> DEBUG - Using default 'AsynchronousPool.QueueSize=5'
> DEBUG - Using default 'AsynchronousPool.KeepAliveTime=60 SECONDS'
> DEBUG - Using default 'AsynchronousPool.AllowCoreThreadTimeOut=true'
> DEBUG - Using default 'AsynchronousPool.QueueType=linked'. Possible values
> are: array, linked, priority, synchronous
> DEBUG - Using default 'AsynchronousPool.OfferTimeout=30 SECONDS'
> DEBUG - Using default 'AsynchronousPool.ShutdownWaitDuration=1 MINUTES'
>
>
> What's the AsynchronousPool ???
>
>
> Please help me to sort this issue?
> I'm running into this issues in production.
>
>
>
> thanks
> Kalyan
>
>
>
>
>
>
>
> --
> Sent from:
> http://tomee-openejb.979440.n4.nabble.com/TomEE-Users-f979441.html
>