You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Glenn Nielsen <gl...@voyager.apg.more.net> on 2002/02/22 17:40:00 UTC
commons dbcp or pool problems
I have been using Tomcat 4.1-dev and its new DBCP DataSource Factory
which uses commons-dbcp, which then uses other commons components
like commons-pool.
Everything works fine, then all of a sudden GenericPool can no longer
get a db connection from the pool. All attempts to get a connection
from the pool fail. The only way to fix the problem is to stop and
restart Tomcat. In case you ask, the db is running fine, and an
application which uses Torque to manage db connections continues
to work. Only the commons-dbcp exhibits the failure.
This has happen twice in the last 24 hours, both times were about 10 - 12
hours after Tomcat started up. The first run was during the day with higher
traffic, the second was overnight with lower traffic. During each run the
DBCP has handled many thousands of connection requests.
Here is a snippet from the exception:
2002-02-22 08:47:53 StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw
exception
java.util.NoSuchElementException: Timeout waiting for idle object
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(Unknown Source)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(Unknown Source)
at
org.apache.taglibs.dbtags.connection.ConnectionTag.doEndTag(ConnectionTag.java:214)
at org.apache.jsp.pastnumsearch$jsp._jspService(pastnumsearch$jsp.java:116)
I am using the following commons components.
commons-collections V1.0 release from July 14, 2001.
commons-pool nightly from Feb 1, 2002.
commons-dbcp nightly from Feb 1, 2002.
Does anyone know of any bugs or changes in any of these which may
be causing the general failure of pooling the dbcp connections?
Thanks,
Glenn
----------------------------------------------------------------------
Glenn Nielsen glenn@more.net | /* Spelin donut madder |
MOREnet System Programming | * if iz ina coment. |
Missouri Research and Education Network | */ |
----------------------------------------------------------------------
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: commons dbcp or pool problems
Posted by Martin Cooper <ma...@tumbleweed.com>.
----- Original Message -----
From: "Geir Magnusson Jr." <ge...@optonline.net>
To: "Jakarta Commons Developers List" <co...@jakarta.apache.org>
Sent: Wednesday, March 06, 2002 12:16 PM
Subject: Re: commons dbcp or pool problems
> On 3/6/02 2:53 PM, "John McNally" <jm...@collab.net> wrote:
>
> >
> > The code in sandbox/jdbc2pool uses the commons pool and DBCP's
> > PreparedStatement pool while attempting to implement the jdbc2
> > specification. Geir and anyone else at jakarta who wants to work on a
> > connection pool, why not join me in finishing this? Geir, why are you
> > saying you want to work on Poolman, what does it have that dbcp does
> > not?
>
> I am saying that I think Poolman is worth saving. I rely on it in several
> client projects, and I am sure there are many users of Poolman here,
despite
> the existence of the dbcp.
>
> This is not a reflection on DBCP, but rather acting on the knowledge that
> Poolman has a huge user base and a long track record of reliability.
I agree that Poolman is worth saving. From what I can tell from various
mailing lists I am on, it has a substantial and enthusiastic following. A
big part of that is because it's recognised as being reliable, good quality
software. IMHO, that makes it a good candidate for Jakarta. ;-)
> So far, the only feedback about this idea has been negative, so at the
> moment, I am pretty convinced it isn't going to happen. That doesn't mean
I
> won't try elsewhere :)
I would support bringing Poolman here. Although I'm far from an expert on
connection pools, I'm also willing to sign up as an initial committer to
help with that process. Just let me know what I can do to help.
--
Martin Cooper
>
> >
> > Yes jakarta developers should be able to work on what they want to work
> > on, but the founding projects of the commons was the pool and dbcp.
>
> Yep - certainly was my motivation.
>
> > Yes
> > it is possible to have multiple implementations of these.
>
> Yep.
>
> [SNIP]
> > Isn't the point of the
> > commons to not have multiple implementations of the same concept?
>
> Not necessarily...
>
>
>
> > "Geir Magnusson Jr." wrote:
> >>
> >> On 3/6/02 6:32 AM, "Ted Husted" <hu...@apache.org> wrote:
> >>
> >>> The only point to anything at Jakarta is that the volunteers will work
> >>> on what they ~want~ to work on. If there are volunteers here who want
to
> >>> work on the dbcp, then we will continue to have a dbcp. If there are
> >>> volunteers here who want to bring Poolman here, then they could do
that
> >>> too.
> >>>
> >>> My only point is that the opportunity now exists; it's just a matter
of
> >>> whether anyone interested in working on connection pools is interested
> >>> in pursuing the Poolman codebase.
> >>
> >> I've talked to Sean every now and then about bringing Poolman here to
> >> Jakarta, and I have another query on this in to him now.
> >>
> >> I'm going to try and do it.
> >>
> >> geir
> >>
> >> --
> >> Geir Magnusson Jr.
geirm@optonline.net
> >> System and Software Consulting
> >> The bytecodes are language independent. - Sam Ruby
> >>
> >> --
> >> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> >> For additional commands, e-mail:
<ma...@jakarta.apache.org>
> >
> > --
> > To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> > For additional commands, e-mail:
<ma...@jakarta.apache.org>
> >
>
> --
> Geir Magnusson Jr. geirm@optonline.net
> System and Software Consulting
> POC lives!
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: commons dbcp or pool problems
Posted by "Geir Magnusson Jr." <ge...@optonline.net>.
On 3/6/02 2:53 PM, "John McNally" <jm...@collab.net> wrote:
>
> The code in sandbox/jdbc2pool uses the commons pool and DBCP's
> PreparedStatement pool while attempting to implement the jdbc2
> specification. Geir and anyone else at jakarta who wants to work on a
> connection pool, why not join me in finishing this? Geir, why are you
> saying you want to work on Poolman, what does it have that dbcp does
> not?
I am saying that I think Poolman is worth saving. I rely on it in several
client projects, and I am sure there are many users of Poolman here, despite
the existence of the dbcp.
This is not a reflection on DBCP, but rather acting on the knowledge that
Poolman has a huge user base and a long track record of reliability.
So far, the only feedback about this idea has been negative, so at the
moment, I am pretty convinced it isn't going to happen. That doesn't mean I
won't try elsewhere :)
>
> Yes jakarta developers should be able to work on what they want to work
> on, but the founding projects of the commons was the pool and dbcp.
Yep - certainly was my motivation.
> Yes
> it is possible to have multiple implementations of these.
Yep.
[SNIP]
> Isn't the point of the
> commons to not have multiple implementations of the same concept?
Not necessarily...
> "Geir Magnusson Jr." wrote:
>>
>> On 3/6/02 6:32 AM, "Ted Husted" <hu...@apache.org> wrote:
>>
>>> The only point to anything at Jakarta is that the volunteers will work
>>> on what they ~want~ to work on. If there are volunteers here who want to
>>> work on the dbcp, then we will continue to have a dbcp. If there are
>>> volunteers here who want to bring Poolman here, then they could do that
>>> too.
>>>
>>> My only point is that the opportunity now exists; it's just a matter of
>>> whether anyone interested in working on connection pools is interested
>>> in pursuing the Poolman codebase.
>>
>> I've talked to Sean every now and then about bringing Poolman here to
>> Jakarta, and I have another query on this in to him now.
>>
>> I'm going to try and do it.
>>
>> geir
>>
>> --
>> Geir Magnusson Jr. geirm@optonline.net
>> System and Software Consulting
>> The bytecodes are language independent. - Sam Ruby
>>
>> --
>> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
>> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
Geir Magnusson Jr. geirm@optonline.net
System and Software Consulting
POC lives!
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: commons dbcp or pool problems
Posted by John McNally <jm...@collab.net>.
Why do you want to bring Poolman here? Commons already has a pool and
connection pool. In an attempt to allow jakarta-turbine-torque to use
any jdbc2 pool, I started looking for an open-source implementation and
found none. Poolman and DBCP can be said to have the possibility of
presenting a DataSource front end, but neither attempt to implement the
specification fully. As PreparedStatement pooling is not addressed
until jdbc3, you could say they are both ahead of the curve.
But pushing either of these without attempting to bring them up to at
least jdbc2 compliance does not make much sense. I have been working on
this for over a month (intermittently, of course) and have received one
code submission from another developer, not a commons committer. Craig
has submitted a jdbc2 style DataSource (meaning bean-like configuration)
example to the DBCP codebase that I think serves as a good example on
how to use the DBCP codebase in an easy to configure manner. It does
not include much configurability and leaves out statement pooling,
however, and it still does not hook into the correct backend for a jdbc2
pool.
The code in sandbox/jdbc2pool uses the commons pool and DBCP's
PreparedStatement pool while attempting to implement the jdbc2
specification. Geir and anyone else at jakarta who wants to work on a
connection pool, why not join me in finishing this? Geir, why are you
saying you want to work on Poolman, what does it have that dbcp does
not?
Yes jakarta developers should be able to work on what they want to work
on, but the founding projects of the commons was the pool and dbcp. Yes
it is possible to have multiple implementations of these. We have an
object pool in turbine that we are still deciding on whether to keep it
maintained or dump it for the current commons pool. We had our own dbcp
as well, which I do not think anyone wants to maintain as a separate
entity even though it works perfectly well. Isn't the point of the
commons to not have multiple implementations of the same concept? I
have to say I still do not quite understand what the commons is.
john mcnally
"Geir Magnusson Jr." wrote:
>
> On 3/6/02 6:32 AM, "Ted Husted" <hu...@apache.org> wrote:
>
> > The only point to anything at Jakarta is that the volunteers will work
> > on what they ~want~ to work on. If there are volunteers here who want to
> > work on the dbcp, then we will continue to have a dbcp. If there are
> > volunteers here who want to bring Poolman here, then they could do that
> > too.
> >
> > My only point is that the opportunity now exists; it's just a matter of
> > whether anyone interested in working on connection pools is interested
> > in pursuing the Poolman codebase.
>
> I've talked to Sean every now and then about bringing Poolman here to
> Jakarta, and I have another query on this in to him now.
>
> I'm going to try and do it.
>
> geir
>
> --
> Geir Magnusson Jr. geirm@optonline.net
> System and Software Consulting
> The bytecodes are language independent. - Sam Ruby
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: commons dbcp or pool problems
Posted by "Geir Magnusson Jr." <ge...@optonline.net>.
On 3/6/02 6:32 AM, "Ted Husted" <hu...@apache.org> wrote:
> The only point to anything at Jakarta is that the volunteers will work
> on what they ~want~ to work on. If there are volunteers here who want to
> work on the dbcp, then we will continue to have a dbcp. If there are
> volunteers here who want to bring Poolman here, then they could do that
> too.
>
> My only point is that the opportunity now exists; it's just a matter of
> whether anyone interested in working on connection pools is interested
> in pursuing the Poolman codebase.
I've talked to Sean every now and then about bringing Poolman here to
Jakarta, and I have another query on this in to him now.
I'm going to try and do it.
geir
--
Geir Magnusson Jr. geirm@optonline.net
System and Software Consulting
The bytecodes are language independent. - Sam Ruby
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: commons dbcp or pool problems
Posted by Ted Husted <hu...@apache.org>.
The only point to anything at Jakarta is that the volunteers will work
on what they ~want~ to work on. If there are volunteers here who want to
work on the dbcp, then we will continue to have a dbcp. If there are
volunteers here who want to bring Poolman here, then they could do that
too.
My only point is that the opportunity now exists; it's just a matter of
whether anyone interested in working on connection pools is interested
in pursuing the Poolman codebase.
-Ted.
Bryan Field-Elliot wrote:
> If dbcp is in good shape (which I'm trying to ask about), then is there
> any point in also trying to bring poolman here?
>
> Bryan
>
> On Tue, 2002-03-05 at 21:00, Ted Husted wrote:
>
> So in the event anyone is interested, the Poolman maintainer has
> retired.
>
> "I no longer have time to maintain the PoolMan
> JDBC driver and resource pooling and caching
> utility, and am now seeking a qualified,
> comitted software engineer to assume control
> over its future. If interested, please send email
> to neville@codestudio.com with a brief summary
> of your qualifications."
>
> http://sourceforge.net/forum/forum.php?forum_id=44181
>
> This really isn't my realm, but I thought I would bring it up in case
> any one here would be up to working with Neville to bring poolman here.
> There was some talk about that last year, so maybe it's time to revive
> the discussions. Of course, we'd have to get the license changed, et
> cetera.
>
> Meanwhile, I'm tucking the last production release away for safekeeping.
>
> -Ted.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: commons dbcp or pool problems
Posted by Bryan Field-Elliot <br...@netmeme.org>.
What is the status of commons dbcp?
Apart from what look like minor edits from Craig, it's been 10 months
(according to CVS) since any major work has been done. Is this
considered done and production quality or it still in beta (or less?)
I noticed that it's part of the Struts nightly build now.. Does that
vouch for its completeness in any way?
If dbcp is in good shape (which I'm trying to ask about), then is there
any point in also trying to bring poolman here?
Bryan
On Tue, 2002-03-05 at 21:00, Ted Husted wrote:
So in the event anyone is interested, the Poolman maintainer has
retired.
"I no longer have time to maintain the PoolMan
JDBC driver and resource pooling and caching
utility, and am now seeking a qualified,
comitted software engineer to assume control
over its future. If interested, please send email
to neville@codestudio.com with a brief summary
of your qualifications."
http://sourceforge.net/forum/forum.php?forum_id=44181
This really isn't my realm, but I thought I would bring it up in case
any one here would be up to working with Neville to bring poolman here.
There was some talk about that last year, so maybe it's time to revive
the discussions. Of course, we'd have to get the license changed, et
cetera.
Meanwhile, I'm tucking the last production release away for safekeeping.
-Ted.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: commons dbcp or pool problems
Posted by Ted Husted <hu...@apache.org>.
So in the event anyone is interested, the Poolman maintainer has
retired.
"I no longer have time to maintain the PoolMan
JDBC driver and resource pooling and caching
utility, and am now seeking a qualified,
comitted software engineer to assume control
over its future. If interested, please send email
to neville@codestudio.com with a brief summary
of your qualifications."
http://sourceforge.net/forum/forum.php?forum_id=44181
This really isn't my realm, but I thought I would bring it up in case
any one here would be up to working with Neville to bring poolman here.
There was some talk about that last year, so maybe it's time to revive
the discussions. Of course, we'd have to get the license changed, et
cetera.
Meanwhile, I'm tucking the last production release away for safekeeping.
-Ted.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: commons dbcp or pool problems
Posted by James House <ja...@interobjective.com>.
At 2/22/2002 08:47 PM +0100, Juozas Baliuka wrote:
>It is very dificult to find pooling related bugs in application.
>May be it has meaning to implement something like "DebugPool" to help find
>problems
>in users code.
>I can think about this, if this idea is interesting.
A few months back I made my own hacks to DBCP in order to have it find
places in our code that didn't free up DB resources properly.
I was able to generate class names and line-numbers (stack trace) for every
place in the code that a statement or result set was created but never
closed, and also able to track where connections were borrowed and never
returned.
All of this was pretty easy to do (an hour's work or so), and only required
mods to the dbcp classes.
If anyone's interested, I could try digging it up, and posting it.
James
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: commons dbcp or pool problems
Posted by Juozas Baliuka <ba...@centras.lt>.
<SKIP>
> Then again, I haven't looked at the pool component, so I don't know
> whether this would be tye symptom for that problem or not. Take what I
> say with a grain of salt. Although it might be a good place to start
> looking.
>
> regards,
> michael
>
<SKIP>
Hi,
It is very dificult to find pooling related bugs in application.
May be it has meaning to implement something like "DebugPool" to help find
problems
in users code.
I can think about this, if this idea is interesting.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: commons dbcp or pool problems
Posted by Glenn Nielsen <gl...@voyager.apg.more.net>.
"Michael A. Smith" wrote:
>
> On Fri, 22 Feb 2002, Glenn Nielsen wrote:
> > I have been using Tomcat 4.1-dev and its new DBCP DataSource Factory
> > which uses commons-dbcp, which then uses other commons components
> > like commons-pool.
> >
> > Everything works fine, then all of a sudden GenericPool can no longer
> > get a db connection from the pool. All attempts to get a connection
> > from the pool fail. The only way to fix the problem is to stop and
> > restart Tomcat. In case you ask, the db is running fine, and an
> > application which uses Torque to manage db connections continues
> > to work. Only the commons-dbcp exhibits the failure.
> >
> > This has happen twice in the last 24 hours, both times were about 10 - 12
> > hours after Tomcat started up. The first run was during the day with higher
> > traffic, the second was overnight with lower traffic. During each run the
> > DBCP has handled many thousands of connection requests.
> >
> > Here is a snippet from the exception:
> >
> > 2002-02-22 08:47:53 StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw
> > exception
> > java.util.NoSuchElementException: Timeout waiting for idle object
> > at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(Unknown Source)
> > at org.apache.commons.dbcp.PoolingDataSource.getConnection(Unknown Source)
> > at
> > org.apache.taglibs.dbtags.connection.ConnectionTag.doEndTag(ConnectionTag.java:214)
> > at org.apache.jsp.pastnumsearch$jsp._jspService(pastnumsearch$jsp.java:116)
> >
>
> Without knowing any details about the pool component (yet -- I plan on
> giving it a good look eventually), I suspect that you may have a pool
> leak. Somewhere you're borrowing from the pool and not returning it to
> the pool. Possibly the return of an object is not in a finally block and
> an rare exception causes it to lose one object.
>
> Then again, I haven't looked at the pool component, so I don't know
> whether this would be tye symptom for that problem or not. Take what I
> say with a grain of salt. Although it might be a good place to start
> looking.
>
Yes, that is something I looked at. This dbcp is only used by dbtags
in JSP pages. During these runs a JSP page failed only a couple of times
before the general dbcp connection pool failure started. So this isn't
due to a JSP page failure causing a connection leak.
I am also looking into the JDBC driver and the DbTags JSP tag library.
The JSP Page Author is also going to be reviewing the code for his usage
of DbTags in his JSP pages.
Regards,
Glenn
----------------------------------------------------------------------
Glenn Nielsen glenn@more.net | /* Spelin donut madder |
MOREnet System Programming | * if iz ina coment. |
Missouri Research and Education Network | */ |
----------------------------------------------------------------------
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: commons dbcp or pool problems
Posted by "Michael A. Smith" <mi...@iammichael.org>.
On Fri, 22 Feb 2002, Glenn Nielsen wrote:
> I have been using Tomcat 4.1-dev and its new DBCP DataSource Factory
> which uses commons-dbcp, which then uses other commons components
> like commons-pool.
>
> Everything works fine, then all of a sudden GenericPool can no longer
> get a db connection from the pool. All attempts to get a connection
> from the pool fail. The only way to fix the problem is to stop and
> restart Tomcat. In case you ask, the db is running fine, and an
> application which uses Torque to manage db connections continues
> to work. Only the commons-dbcp exhibits the failure.
>
> This has happen twice in the last 24 hours, both times were about 10 - 12
> hours after Tomcat started up. The first run was during the day with higher
> traffic, the second was overnight with lower traffic. During each run the
> DBCP has handled many thousands of connection requests.
>
> Here is a snippet from the exception:
>
> 2002-02-22 08:47:53 StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw
> exception
> java.util.NoSuchElementException: Timeout waiting for idle object
> at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(Unknown Source)
> at org.apache.commons.dbcp.PoolingDataSource.getConnection(Unknown Source)
> at
> org.apache.taglibs.dbtags.connection.ConnectionTag.doEndTag(ConnectionTag.java:214)
> at org.apache.jsp.pastnumsearch$jsp._jspService(pastnumsearch$jsp.java:116)
>
Without knowing any details about the pool component (yet -- I plan on
giving it a good look eventually), I suspect that you may have a pool
leak. Somewhere you're borrowing from the pool and not returning it to
the pool. Possibly the return of an object is not in a finally block and
an rare exception causes it to lose one object.
Then again, I haven't looked at the pool component, so I don't know
whether this would be tye symptom for that problem or not. Take what I
say with a grain of salt. Although it might be a good place to start
looking.
regards,
michael
> I am using the following commons components.
>
> commons-collections V1.0 release from July 14, 2001.
> commons-pool nightly from Feb 1, 2002.
> commons-dbcp nightly from Feb 1, 2002.
>
> Does anyone know of any bugs or changes in any of these which may
> be causing the general failure of pooling the dbcp connections?
>
> Thanks,
>
> Glenn
>
> ----------------------------------------------------------------------
> Glenn Nielsen glenn@more.net | /* Spelin donut madder |
> MOREnet System Programming | * if iz ina coment. |
> Missouri Research and Education Network | */ |
> ----------------------------------------------------------------------
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>