[jira] [Commented] (AIRAVATA-3186) ThriftClientPool$ThriftClientException: Could not get a resource from the pool

["Marcus Christie (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRAVATA-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957997#comment-16957997 ] 

Marcus Christie commented on AIRAVATA-3186:
-------------------------------------------

I'm going to upgrade to commons-pool2 which has the feature of logging stacktraces of objects that were retrieved from the pool but then never returned, what they call abandoned objects.

{quote}
The pool can also be configured to detect and remove "abandoned" objects, i.e. objects that have been checked out of the pool but neither used nor returned before the configured removeAbandonedTimeout. Abandoned object removal can be configured to happen when borrowObject is invoked and the pool is close to starvation, or it can be executed by the idle object evictor, or both. If pooled objects implement the TrackedUse interface, their last use will be queried using the getLastUsed method on that interface; otherwise abandonment is determined by how long an object has been checked out from the pool.
{quote}

https://commons.apache.org/proper/commons-pool/api-2.7.0/index.html


> ThriftClientPool$ThriftClientException: Could not get a resource from the pool
> ------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-3186
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-3186
>             Project: Airavata
>          Issue Type: Bug
>          Components: Airavata API
>            Reporter: Marcus Christie
>            Assignee: Marcus Christie
>            Priority: Major
>
> {noformat}
> 2019-07-26 16:02:39,871 [pool-24-thread-28] ERROR o.a.t.server.TThreadPoolServer  - Error occurred during processing of message.
> org.apache.airavata.common.utils.ThriftClientPool$ThriftClientException: Could not get a resource from the pool
>         at org.apache.airavata.common.utils.ThriftClientPool.getResource(ThriftClientPool.java:144)
>         at org.apache.airavata.api.server.handler.AiravataServerHandler.userHasAccess(AiravataServerHandler.java:5370)
>         at org.apache.airavata.service.security.interceptor.SecurityInterceptor.invoke(SecurityInterceptor.java:56)
>         at org.apache.airavata.api.Airavata$Processor$userHasAccess.getResult(Airavata.java:22848)
>         at org.apache.airavata.api.Airavata$Processor$userHasAccess.getResult(Airavata.java:22832)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>         at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.NoSuchElementException: Timeout waiting for idle object
>         at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1174)
>         at org.apache.airavata.common.utils.ThriftClientPool.getResource(ThriftClientPool.java:132)
>         ... 10 common frames omitted
> {noformat}
> h5. TODO
> - [ ] add properties for configuring abandoned object removal. There's some overhead so probably don't want this turned on in production.
> - [ ] Resource leak warning here: https://github.com/apache/airavata/blob/staging/modules/commons/src/main/java/org/apache/airavata/common/utils/ThriftClientPool.java#L111



--
This message was sent by Atlassian Jira
(v8.3.4#803005)