You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Christopher Schultz <ch...@christopherschultz.net> on 2015/08/02 03:14:17 UTC
Re: [OT] "Too Many Connections" exceptions after moving to Tomcat 8
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Phil,
On 8/1/15 11:12 AM, Phil Steitz wrote:
> Sorry the docs are not that clear. The problem is that the config
> properties work together to determine behavior and documenting
> them individually makes it hard to put the whole picture together.
> Improvement patches most welcome :) In any case, here is how it
> works:
>
> Connections are evaluated for abandonment when they are out in
> circulation - checked out to clients. If you have set
> timeBetweenEvictionRunsMillis to a positive value, pool
> maintenance runs every timeBetweenEvictionRunsMillis milliseconds.
> If you have removeAbandonedOnMaintenance set to true, each time
> maintenance runs the pool removes abandoned connections. If you
> have removeAbandonedOnBorrow set to true, the pool removes
> abandoned connections if it is nearing depletion when a borrow
> request arrives. In both cases, the pool looks at the statistics
> that it maintains on the list of all objects checked out by clients
> to determine which ones appear to be abandoned. To appear
> abandoned means to be checked out by a client but not used for
> longer than the removeAbandonedTimeout. For DBCP, "used" means the
> connection has been used.
So when during all that does the abandoned connection get "logged"?
If I have a pool with two connections in it, and the first one gets
leaked (after, lets say, 10 seconds) but the second one is returned
properly, and the application continues to borrow and return the
second connection, what happens?
Are you only checking connections for "abandonment" after they are
returned and then when a client attempts to borrow that specific
connection? Or, are all currently-borrowed connections checked for
abandonment whenever any borrow operation is requested?
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
iQIcBAEBCAAGBQJVvW7pAAoJEBzwKT+lPKRYY0UQAIBXTrkBRNC9j7KBYw8DZFcJ
UY+U1WrHi7osxEseupCjSjP/xmmOHoC16LkEXqtil5k8rd5yQ9D5I1imPKYZSjiE
GBBkkjrbvlwDq/sRp3fxN8La6ueTed4yyh96J5DFfx0opcTtX06YMnpkJDTN2iTt
R3GUKbBo4dNZKIFNGhicR4j9JbSwerxbfJDVpEtb0igO7fQu1dRTDQh7gijyUsZ7
PPa0owq9WOBl5dr4ebsoipirArbLbB4WZqCtteOj5QpafEOYyGuKAjFs6PhodVAR
A2BlikA9+s6AUcKVnqP+CabZFNYO/1raZWl61q7vSWNR9YMrSSI6TapCsBgzNDQW
p8KloJYqjaZFNGtVW23eDtRjTehABnQqoXrB4OL2CEFqgZVjnfpCDl1K0b8wc01y
QNnA0eo40462XkwsRSyMOwtunLu02mGayOrzrn7blJ0f2Kj594+/U6HKqAttRlOX
0Sr2VnYJTaBRACHVjycKjjQh1A1lIIfRoIQ0enMBiw8kZ4rVpQsCvwk/5h54/oap
5osEzulFO7zwqhAxWaHigFG9hCeHJHGr7eT4XUXfUqw+/zg6vI//r5rLExIwSi4D
p/R214PYSkEMiz+i4uSu+sUOYpOJbaAnJH0R6kXSKkoDmaaylTyLuHujPvLKPF6b
ZV739LJI+k5cgcDEWmjv
=PKZ/
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] "Too Many Connections" exceptions after moving to Tomcat
8
Posted by Phil Steitz <ph...@gmail.com>.
On 8/4/15 2:03 PM, Christopher Schultz wrote:
> Phil,
>
> On 8/1/15 9:34 PM, Phil Steitz wrote:
> > On 8/1/15 6:14 PM, Christopher Schultz wrote:
> >> Phil,
> >>
> >> On 8/1/15 11:12 AM, Phil Steitz wrote:
> >>> Sorry the docs are not that clear. The problem is that the
> >>> config properties work together to determine behavior and
> >>> documenting them individually makes it hard to put the whole
> >>> picture together. Improvement patches most welcome :) In any
> >>> case, here is how it works:
> >>
> >>> Connections are evaluated for abandonment when they are out in
> >>> circulation - checked out to clients. If you have set
> >>> timeBetweenEvictionRunsMillis to a positive value, pool
> >>> maintenance runs every timeBetweenEvictionRunsMillis
> >>> milliseconds. If you have removeAbandonedOnMaintenance set to
> >>> true, each time maintenance runs the pool removes abandoned
> >>> connections. If you have removeAbandonedOnBorrow set to true,
> >>> the pool removes abandoned connections if it is nearing
> >>> depletion when a borrow request arrives. In both cases, the
> >>> pool looks at the statistics that it maintains on the list of
> >>> all objects checked out by clients to determine which ones
> >>> appear to be abandoned. To appear abandoned means to be
> >>> checked out by a client but not used for longer than the
> >>> removeAbandonedTimeout. For DBCP, "used" means the connection
> >>> has been used.
> >>
> >> So when during all that does the abandoned connection get
> >> "logged"?
>
> > The log message is written when the abandoned connection is closed
> > by the pool.
>
> So if the client code never calls close(), does that mean the
> amandonment is never logged? IIRC, DBCP 1.x would log the stack trace
> as soon as removeAbandonedTimeout elapsed if the connection hadn't
> been returned.
The behavior is the same in 2.x as 1.x with the exception that in
2.x, pool maintenance can also trigger abandoned object removal. In
1.x, it is necessary to attempt a borrow to get abandoned
connections to be removed. Both 1.x and 2.x essentially remove
abandoned connections in batches, closing the connections and
logging the traces when the batches run. In 1.x, the only way to
trigger an abandoned cleanup batch is to request a connection when
the pool is near depletion. In 2.x, if removeAbandonedOnMaintenance
is true, an abandoned cleanup batch will run every
timeBetweenEvictionRunsMillis ms.
>
> > (The pool bug was basically delaying the effective write; but the
> > stack trace goes to the buffer when the close happens)
> >>
> >> If I have a pool with two connections in it, and the first one
> >> gets leaked (after, lets say, 10 seconds) but the second one is
> >> returned properly, and the application continues to borrow and
> >> return the second connection, what happens?
>
> > That depends on the settings. See below.
> >>
> >> Are you only checking connections for "abandonment" after they
> >> are returned and then when a client attempts to borrow that
> >> specific connection? Or, are all currently-borrowed connections
> >> checked for abandonment whenever any borrow operation is
> >> requested?
>
> > It is the latter, assuming removeAbandonedOnBorrow is true.
> > Removal on borrow also requires that the pool be near depletion
> > when the borrow happens (see the javadoc for precise description
> > of conditions). There are two things that can trigger abandoned
> > object removal - a borrow request and pool maintenance. In each
> > case, if the config setting to enable the check on that event is
> > on, all checked out objects are examined. The ones that appear
> > abandoned are destroyed (in DBCP that means physical connections
> > are closed).
>
> Why is there no option to dump a warning as soon as the timeout
> elapses?
Ideas / patches welcome on how to do that efficiently.
> I suppose if the pool is being used slightly more than
> /never/, the abandoned notice will happen pretty quickly, but for
> testing purposes, it means that a single-user will have to perform
> some operation, wait, and then perform another operation in order to
> get the feedback about the abandonment. That's surprising behavior if
> you ask me.
See above. If you set removeAbandonedOnMaintenance and
maxTimeBetweenEvictionRunsMillis, abandoned connection removal will
happen regularly.
Phil
>
> -chris
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] "Too Many Connections" exceptions after moving to Tomcat 8
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Phil,
On 8/1/15 9:34 PM, Phil Steitz wrote:
> On 8/1/15 6:14 PM, Christopher Schultz wrote:
>> Phil,
>>
>> On 8/1/15 11:12 AM, Phil Steitz wrote:
>>> Sorry the docs are not that clear. The problem is that the
>>> config properties work together to determine behavior and
>>> documenting them individually makes it hard to put the whole
>>> picture together. Improvement patches most welcome :) In any
>>> case, here is how it works:
>>
>>> Connections are evaluated for abandonment when they are out in
>>> circulation - checked out to clients. If you have set
>>> timeBetweenEvictionRunsMillis to a positive value, pool
>>> maintenance runs every timeBetweenEvictionRunsMillis
>>> milliseconds. If you have removeAbandonedOnMaintenance set to
>>> true, each time maintenance runs the pool removes abandoned
>>> connections. If you have removeAbandonedOnBorrow set to true,
>>> the pool removes abandoned connections if it is nearing
>>> depletion when a borrow request arrives. In both cases, the
>>> pool looks at the statistics that it maintains on the list of
>>> all objects checked out by clients to determine which ones
>>> appear to be abandoned. To appear abandoned means to be
>>> checked out by a client but not used for longer than the
>>> removeAbandonedTimeout. For DBCP, "used" means the connection
>>> has been used.
>>
>> So when during all that does the abandoned connection get
>> "logged"?
>
> The log message is written when the abandoned connection is closed
> by the pool.
So if the client code never calls close(), does that mean the
amandonment is never logged? IIRC, DBCP 1.x would log the stack trace
as soon as removeAbandonedTimeout elapsed if the connection hadn't
been returned.
> (The pool bug was basically delaying the effective write; but the
> stack trace goes to the buffer when the close happens)
>>
>> If I have a pool with two connections in it, and the first one
>> gets leaked (after, lets say, 10 seconds) but the second one is
>> returned properly, and the application continues to borrow and
>> return the second connection, what happens?
>
> That depends on the settings. See below.
>>
>> Are you only checking connections for "abandonment" after they
>> are returned and then when a client attempts to borrow that
>> specific connection? Or, are all currently-borrowed connections
>> checked for abandonment whenever any borrow operation is
>> requested?
>
> It is the latter, assuming removeAbandonedOnBorrow is true.
> Removal on borrow also requires that the pool be near depletion
> when the borrow happens (see the javadoc for precise description
> of conditions). There are two things that can trigger abandoned
> object removal - a borrow request and pool maintenance. In each
> case, if the config setting to enable the check on that event is
> on, all checked out objects are examined. The ones that appear
> abandoned are destroyed (in DBCP that means physical connections
> are closed).
Why is there no option to dump a warning as soon as the timeout
elapses? I suppose if the pool is being used slightly more than
/never/, the abandoned notice will happen pretty quickly, but for
testing purposes, it means that a single-user will have to perform
some operation, wait, and then perform another operation in order to
get the feedback about the abandonment. That's surprising behavior if
you ask me.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
iQIcBAEBCAAGBQJVwSifAAoJEBzwKT+lPKRYoaAP/jLgMP3BWm3j5OTvAFw55Sbm
GyDQel0X0LxlXC1Nn/YETjTvUGh3d1sv0FE0zyuNFfrmEGi35hX6vEIFA+aAx5wb
1LjUR4gUTbMR/4mYGbVyHb65Pt+7CY/oMuGbp22o+2DwXEy/+1sS7lHAXK/xKkbb
F+HcLN7bU2khLp/1JuSp7NfmSEpBKkpuitkTUyGR27ijorYRRqpLfM80/aJV8aTh
MSMhgFuJ6igkJv29oTMqTXY3k+xIfZeIpJuYQ0yFLFwKmZuxPgSP+9hSfOyQAbDs
fev2bcwE9yYmypWOJrPBFR+T7xt61BSLzi1LwUixuTDP6eZEOYrYDvXd0lrB+Ohv
jS5Md182lWiAwbF+8grMwALDTUCE4cCfq/BqO1gSWNaZSDJoFhmm0QmPPgB2iVU+
ZEX9rkwCfY0SKal2x7uJzMwCJYgtPlXq88Fr2aX+Wvg90nXKqwVucAMBf+zOyaP3
tEIuw4H20ug6OPgbfUmdKS1qYLyjS5V7XzP/LoZCNjzDK+pqrmdkRneNdSiIlTzv
2HIQKtIYFZXHEMOmnBGiMWrGfF8lUrAk/rpmFerXk2RI/e6H9ufAExSIz7xhKZEG
AToWrGhb7DOCBOAsABoGnU4R9qE27FuWBcceKdDmSRZzqThmT+9o2tLjXEGJfcc1
Bmt9TxMpWJYKboZkaj/w
=eTpv
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] "Too Many Connections" exceptions after moving to Tomcat
8
Posted by Phil Steitz <ph...@gmail.com>.
On 8/1/15 6:14 PM, Christopher Schultz wrote:
> Phil,
>
> On 8/1/15 11:12 AM, Phil Steitz wrote:
> > Sorry the docs are not that clear. The problem is that the config
> > properties work together to determine behavior and documenting
> > them individually makes it hard to put the whole picture together.
> > Improvement patches most welcome :) In any case, here is how it
> > works:
>
> > Connections are evaluated for abandonment when they are out in
> > circulation - checked out to clients. If you have set
> > timeBetweenEvictionRunsMillis to a positive value, pool
> > maintenance runs every timeBetweenEvictionRunsMillis milliseconds.
> > If you have removeAbandonedOnMaintenance set to true, each time
> > maintenance runs the pool removes abandoned connections. If you
> > have removeAbandonedOnBorrow set to true, the pool removes
> > abandoned connections if it is nearing depletion when a borrow
> > request arrives. In both cases, the pool looks at the statistics
> > that it maintains on the list of all objects checked out by clients
> > to determine which ones appear to be abandoned. To appear
> > abandoned means to be checked out by a client but not used for
> > longer than the removeAbandonedTimeout. For DBCP, "used" means the
> > connection has been used.
>
> So when during all that does the abandoned connection get "logged"?
The log message is written when the abandoned connection is closed
by the pool. (The pool bug was basically delaying the effective
write; but the stack trace goes to the buffer when the close happens)
>
> If I have a pool with two connections in it, and the first one gets
> leaked (after, lets say, 10 seconds) but the second one is returned
> properly, and the application continues to borrow and return the
> second connection, what happens?
That depends on the settings. See below.
>
> Are you only checking connections for "abandonment" after they are
> returned and then when a client attempts to borrow that specific
> connection? Or, are all currently-borrowed connections checked for
> abandonment whenever any borrow operation is requested?
It is the latter, assuming removeAbandonedOnBorrow is true. Removal
on borrow also requires that the pool be near depletion when the
borrow happens (see the javadoc for precise description of
conditions). There are two things that can trigger abandoned object
removal - a borrow request and pool maintenance. In each case, if
the config setting to enable the check on that event is on, all
checked out objects are examined. The ones that appear abandoned
are destroyed (in DBCP that means physical connections are closed).
Phil
>
> Thanks,
> -chris
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org