You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by prabhu Mahendran <pr...@gmail.com> on 2017/03/22 04:43:56 UTC
Cannot get a connection, pool error Timeout waiting for idle object
in PutSQL?
I have increased the concurrent tasks to be '10' for PutSQL processor.
At that time it shows below error but there is no data loss.
*failed to process due to
org.apache.nifi.processor.exception.ProcessException:
org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool
error Timeout waiting for idle object; rolling back session: *
*if i have remove concurrent tasks then it worked without those exception*
while google this exception i have found answer in below link
I am getting Cannot get a connection, pool error Timeout waiting for idle
object, When I try to create more than 250 threads in my web application
But i don't know how to avoid this issue in NiFi putSQL.
Can anyone help me to resolve this?
Re: Cannot get a connection, pool error Timeout waiting for idle
object in PutSQL?
Posted by prabhu Mahendran <pr...@gmail.com>.
Matt,
Thanks for your suggestion.It worked.
I have set 4GB as heap memory for NiFi.
And use approximately 30 concurrent tasks in my flow.
It boost up my NiFi flow.But it uses 100% CPU Usage.
If i haven't use concurrent tasks then CPU usage is normal.
Is there is any way to reduce or avoid the CPU Usage?
On Wed, Mar 22, 2017 at 8:05 PM, Matt Burgess <ma...@apache.org> wrote:
> I just noticed this answer on SO as well [1].
>
> Regards,
> Matt
>
> [1] http://stackoverflow.com/questions/42942759/cannot-get-
> a-connection-pool-error-timeout-waiting-for-idle-object-in-putsql
>
> On Wed, Mar 22, 2017 at 10:33 AM, Matt Burgess <ma...@apache.org>
> wrote:
> > Prabhu,
> >
> > What are your settings for the DBCPConnectionPool controller service?
> > The defaults are 8 Max Connections and 500 milliseconds for Max Wait
> > Time. For 10 concurrent PutSQL tasks, the first 8 will likely get
> > connections, and if none are returned in 500 milliseconds, then one of
> > the other tasks will not get a connection, leading to the error you
> > see above.
> >
> > I recommend setting Max Connections as high as is prudent (at least
> > the number of concurrent tasks using the controller service), and
> > perhaps extending to Max Wait Time to 1 second or more, depending on
> > how long you are willing for a task to wait for a connection to be
> > returned to the pool by some other task.
> >
> > Regards,
> > Matt
> >
> >
> > On Wed, Mar 22, 2017 at 12:43 AM, prabhu Mahendran
> > <pr...@gmail.com> wrote:
> >> I have increased the concurrent tasks to be '10' for PutSQL processor.
> >>
> >> At that time it shows below error but there is no data loss.
> >>
> >> failed to process due to
> >> org.apache.nifi.processor.exception.ProcessException:
> >> org.apache.commons.dbcp.SQLNestedException: Cannot get a connection,
> pool
> >> error Timeout waiting for idle object; rolling back session:
> >>
> >> if i have remove concurrent tasks then it worked without those exception
> >>
> >> while google this exception i have found answer in below link
> >>
> >> I am getting Cannot get a connection, pool error Timeout waiting for
> idle
> >> object, When I try to create more than 250 threads in my web application
> >>
> >> But i don't know how to avoid this issue in NiFi putSQL.
> >>
> >> Can anyone help me to resolve this?
> >>
>
Re: Cannot get a connection, pool error Timeout waiting for idle
object in PutSQL?
Posted by Matt Burgess <ma...@apache.org>.
I just noticed this answer on SO as well [1].
Regards,
Matt
[1] http://stackoverflow.com/questions/42942759/cannot-get-a-connection-pool-error-timeout-waiting-for-idle-object-in-putsql
On Wed, Mar 22, 2017 at 10:33 AM, Matt Burgess <ma...@apache.org> wrote:
> Prabhu,
>
> What are your settings for the DBCPConnectionPool controller service?
> The defaults are 8 Max Connections and 500 milliseconds for Max Wait
> Time. For 10 concurrent PutSQL tasks, the first 8 will likely get
> connections, and if none are returned in 500 milliseconds, then one of
> the other tasks will not get a connection, leading to the error you
> see above.
>
> I recommend setting Max Connections as high as is prudent (at least
> the number of concurrent tasks using the controller service), and
> perhaps extending to Max Wait Time to 1 second or more, depending on
> how long you are willing for a task to wait for a connection to be
> returned to the pool by some other task.
>
> Regards,
> Matt
>
>
> On Wed, Mar 22, 2017 at 12:43 AM, prabhu Mahendran
> <pr...@gmail.com> wrote:
>> I have increased the concurrent tasks to be '10' for PutSQL processor.
>>
>> At that time it shows below error but there is no data loss.
>>
>> failed to process due to
>> org.apache.nifi.processor.exception.ProcessException:
>> org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool
>> error Timeout waiting for idle object; rolling back session:
>>
>> if i have remove concurrent tasks then it worked without those exception
>>
>> while google this exception i have found answer in below link
>>
>> I am getting Cannot get a connection, pool error Timeout waiting for idle
>> object, When I try to create more than 250 threads in my web application
>>
>> But i don't know how to avoid this issue in NiFi putSQL.
>>
>> Can anyone help me to resolve this?
>>
Re: Cannot get a connection, pool error Timeout waiting for idle
object in PutSQL?
Posted by Matt Burgess <ma...@apache.org>.
Prabhu,
What are your settings for the DBCPConnectionPool controller service?
The defaults are 8 Max Connections and 500 milliseconds for Max Wait
Time. For 10 concurrent PutSQL tasks, the first 8 will likely get
connections, and if none are returned in 500 milliseconds, then one of
the other tasks will not get a connection, leading to the error you
see above.
I recommend setting Max Connections as high as is prudent (at least
the number of concurrent tasks using the controller service), and
perhaps extending to Max Wait Time to 1 second or more, depending on
how long you are willing for a task to wait for a connection to be
returned to the pool by some other task.
Regards,
Matt
On Wed, Mar 22, 2017 at 12:43 AM, prabhu Mahendran
<pr...@gmail.com> wrote:
> I have increased the concurrent tasks to be '10' for PutSQL processor.
>
> At that time it shows below error but there is no data loss.
>
> failed to process due to
> org.apache.nifi.processor.exception.ProcessException:
> org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool
> error Timeout waiting for idle object; rolling back session:
>
> if i have remove concurrent tasks then it worked without those exception
>
> while google this exception i have found answer in below link
>
> I am getting Cannot get a connection, pool error Timeout waiting for idle
> object, When I try to create more than 250 threads in my web application
>
> But i don't know how to avoid this issue in NiFi putSQL.
>
> Can anyone help me to resolve this?
>