You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomee.apache.org by jfjames <je...@atosorigin.com> on 2008/06/06 18:10:23 UTC
DataSource configuration for production
I'm doing some tests before putting OpenEJB in production and I'm facing a
problem with the DataSource configuration.
My environment is :
1/ Application deployed on Tomcat 6.0.16 + Open EJB 3.0,
2/ Database : MySQL 4.0.1 + Mysql Connector 5.0.4
3/ JPA provider : Hibernate Entity Manager 3.3.1 (with
hibernate.connection.release_mode parameter set to auto)
4/ Sun JVM 1.5.0 on Windows NT (Linux in production)
5/ DataSource configuration : no Evictor configured
(TimeBetweenEvictionRunsMillis=-1), MaxActive=10 and MaxWait=-1.
My application exposes a JAX-WS interface and I use a benchmark based on a
multi-threaded client injector.
My problem is : when the number of concurrent injector threads exceeds the
maxActive parameter of the DataSource, many database connections are opened.
For instance more than 50 connexions for 10 running threads ... far more
than MaxActive !
I've checked the DBCP and COMMON POOLS code source and normally this should
not happen : when no connection is available from the pool, we should wait
without creating new one (MaxWait=-1). So why are new connections created in
such quantity ?
Has anyone encountered and resolve this kind of problems ? I need to resolve
it before going to production.
Any help appreciated.
--
View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17695975.html
Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: DataSource configuration for production
Posted by Dain Sundstrom <da...@iq80.com>.
I'll commit it later today.
-dain
On Jun 12, 2008, at 2:32 AM, jfjames wrote:
>
> OK, we've opened a DBCP JIRA issue :
> https://issues.apache.org/jira/browse/DBCP-269.
>
> It would be very nice to include this patch in OpenEJB 3.0.1.
>
> -JF
>
>
> David Blevins wrote:
>>
>>
>> On Jun 11, 2008, at 4:44 AM, jfjames wrote:
>>
>>>
>>> We're back ... It seems we’ve identified the cause of the problem.
>>> It is
>>> located in DBCP 1.3. In fact, the isClosed method of the
>>> DelegatingConnection class doesn’t really close the underlying JDBC
>>> connection :
>>> * when called from the destroyObject method of the
>>> PoolableConnectionFactory, the _closed variable is set to false,
>>> * therefore the test if ( _closed || _conn.isClosed() ) doesn’t
>>> allow to
>>> propagate the close along the delegating chain (up to the JDBC
>>> connection).
>>>
>>> According to us, it should be replaced by if ( _closed &&
>>> _conn.isClosed()
>>> ). We’ve done some tests with this patch and it works fine :
>>> maxActive is
>>> never exceeded and the number of connections to the database
>>> server is
>>> stable.
>>>
>>> Now, we have to check the impact on the DBCP jUnit tests before
>>> going
>>> further ...
>>
>> Great work! Going to be a fantastic contribution.
>>
>> Once you're confident on the impact to the dbcp unit tests, craft
>> up a
>> patch file and open up a JIRA for the issue at
>> http://issues.apache.org/jira/browse/DBCP
>> I can help you with instructions on creating patches if you need it
>> (just an "svn diff > myPatch.txt" for the most part). Then I can go
>> tap those guys on the shoulder and let em know we're waiting on it
>> and
>> it's critical for us.
>>
>> If we can get them to commit it we won't need to wait for a DBCP
>> release and can just roll up a custom build that we can use in 3.0.1.
>>
>> -David
>>
>>
>>> jfjames wrote:
>>>>
>>>> We've spent some time today investigating what actually happens in
>>>> the
>>>> DataSource ... Not so easy since DBCP code is a little tricky !
>>>>
>>>> We've observed that JDBC connections which are realeased from the
>>>> pool by
>>>> the Evictor are not physically closed :
>>>> 1/ from the DataSource standpoint : the maximum size of the pool is
>>>> never
>>>> exceeded (numActive is always inferior to maxActive),
>>>> 2/ but from the dataserver standpoint : the number of connections
>>>> is
>>>> always increasing (up to the maximum allowed by the server).
>>>>
>>>> We haven't identified the exact cause of this issue : for some
>>>> unknown
>>>> reason the DelegatingConnection.close() method consider the JDBC
>>>> connection as already closed which is wrong.
>>>>
>>>> Next step tomorrow ...
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17775606.html
>>> Sent from the OpenEJB User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17796021.html
> Sent from the OpenEJB User mailing list archive at Nabble.com.
>
Re: DataSource configuration for production
Posted by David Blevins <da...@visi.com>.
On Jun 12, 2008, at 10:07 AM, jfjames wrote:
>
> In parallel to this DBCP patch, I've developped an OpenEJB service
> in order
> to use C3P0. My objective is to have a fallback solution in
> production in
> case of problem with DBCP. Everything works fine (very good exercice
> to
> understand how OpenEJB configuration works internally ... to be
> honest I
> must admit there is still some mystery around XBean in my
> understanding) ,
> except that no commit/rollback are sent to the database. This may be
> due to
> the fact that C3P0 doesn't support distributed transaction and its
> connections are not enrolled by the TransactionManager (no
> implementation of
> ManagedConnection). There is the same issue with Proxool. Do you
> know any
> alternative to DBCP which could be used with OpenEJB ?
There's the Geronimo connection manager, but I'm not aware of anything
else. DBCP also didn't support any concepts related to transaction
handling either and we had to add all that so we could use it in
OpenEJB. It was one of Dain's mini side efforts that took a few weeks.
Feel free to ask for xbean info of any kind on the dev list.
-David
> jfjames wrote:
>>
>> OK, we've opened a DBCP JIRA issue :
>> https://issues.apache.org/jira/browse/DBCP-269.
>>
>> It would be very nice to include this patch in OpenEJB 3.0.1.
>>
>> -JF
>>
>>
>> David Blevins wrote:
>>>
>>>
>>> On Jun 11, 2008, at 4:44 AM, jfjames wrote:
>>>
>>>>
>>>> We're back ... It seems we’ve identified the cause of the problem.
>>>> It is
>>>> located in DBCP 1.3. In fact, the isClosed method of the
>>>> DelegatingConnection class doesn’t really close the underlying JDBC
>>>> connection :
>>>> * when called from the destroyObject method of the
>>>> PoolableConnectionFactory, the _closed variable is set to false,
>>>> * therefore the test if ( _closed || _conn.isClosed() ) doesn’t
>>>> allow to
>>>> propagate the close along the delegating chain (up to the JDBC
>>>> connection).
>>>>
>>>> According to us, it should be replaced by if ( _closed &&
>>>> _conn.isClosed()
>>>> ). We’ve done some tests with this patch and it works fine :
>>>> maxActive is
>>>> never exceeded and the number of connections to the database
>>>> server is
>>>> stable.
>>>>
>>>> Now, we have to check the impact on the DBCP jUnit tests before
>>>> going
>>>> further ...
>>>
>>> Great work! Going to be a fantastic contribution.
>>>
>>> Once you're confident on the impact to the dbcp unit tests, craft
>>> up a
>>> patch file and open up a JIRA for the issue at
>>> http://issues.apache.org/jira/browse/DBCP
>>> I can help you with instructions on creating patches if you need it
>>> (just an "svn diff > myPatch.txt" for the most part). Then I can go
>>> tap those guys on the shoulder and let em know we're waiting on it
>>> and
>>> it's critical for us.
>>>
>>> If we can get them to commit it we won't need to wait for a DBCP
>>> release and can just roll up a custom build that we can use in
>>> 3.0.1.
>>>
>>> -David
>>>
>>>
>>>> jfjames wrote:
>>>>>
>>>>> We've spent some time today investigating what actually happens in
>>>>> the
>>>>> DataSource ... Not so easy since DBCP code is a little tricky !
>>>>>
>>>>> We've observed that JDBC connections which are realeased from the
>>>>> pool by
>>>>> the Evictor are not physically closed :
>>>>> 1/ from the DataSource standpoint : the maximum size of the pool
>>>>> is
>>>>> never
>>>>> exceeded (numActive is always inferior to maxActive),
>>>>> 2/ but from the dataserver standpoint : the number of
>>>>> connections is
>>>>> always increasing (up to the maximum allowed by the server).
>>>>>
>>>>> We haven't identified the exact cause of this issue : for some
>>>>> unknown
>>>>> reason the DelegatingConnection.close() method consider the JDBC
>>>>> connection as already closed which is wrong.
>>>>>
>>>>> Next step tomorrow ...
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17775606.html
>>>> Sent from the OpenEJB User mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17804966.html
> Sent from the OpenEJB User mailing list archive at Nabble.com.
>
>
Re: DataSource configuration for production
Posted by David Blevins <da...@visi.com>.
On Jun 19, 2008, at 9:14 AM, jfjames wrote:
>
> We've run our tests and check your fix. All is OK, there is no more
> connection leaks without or without evictor. Thank you very much.
Great! We'll roll up another DBCP binary for the upcoming 3.0.1
release.
-David
> jfjames wrote:
>>
>> Thank you Dain for your reactivity. I'm out of my office traveling
>> and
>> having meetings for two days. A colleague has done some tests which
>> confirms that your fix is OK. I've planned to confirm by doing my own
>> tests on Thursday.
>>
>> Dain Sundstrom wrote:
>>>
>>> I think I have finally fixed this issue. Can you retest and let me
>>> know if it is working for you?
>>>
>>> Thanks,
>>>
>>> -dain
>>>
>>> On Jun 16, 2008, at 10:56 AM, Dain Sundstrom wrote:
>>>
>>>> On Jun 13, 2008, at 8:43 AM, jfjames wrote:
>>>>
>>>>> My intention was to find an alternative to DBCP ready to be used
>>>>> in
>>>>> production with OpenEJB. Since it doesn't exist, I have to change
>>>>> my mind.
>>>>> If you agree to commit our patch quickly, I'm OK to use DBCP in
>>>>> production.
>>>>
>>>> Quick status update. I started working on the patch, but it isn't
>>>> as simple as applying it.... When it comes to the
>>>> DelegateConnection
>>>> class, it should appear to the user to be an plain old Connection
>>>> without any special pooling properties. This mean that when the
>>>> DelegateConnection is closed, it must appear to the caller that the
>>>> connection is actually closed, so methods like isClosed should
>>>> return true.
>>>>
>>>> Instead of changing the isClosed method, I'll change the
>>>> reallyClose
>>>> method so it only checks the real connection before closing.
>>>>
>>>> -dain
>>>
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p18012556.html
> Sent from the OpenEJB User mailing list archive at Nabble.com.
>
>
Re: DataSource configuration for production
Posted by jfjames <je...@atosorigin.com>.
We've run our tests and check your fix. All is OK, there is no more
connection leaks without or without evictor. Thank you very much.
jfjames wrote:
>
> Thank you Dain for your reactivity. I'm out of my office traveling and
> having meetings for two days. A colleague has done some tests which
> confirms that your fix is OK. I've planned to confirm by doing my own
> tests on Thursday.
>
> Dain Sundstrom wrote:
>>
>> I think I have finally fixed this issue. Can you retest and let me
>> know if it is working for you?
>>
>> Thanks,
>>
>> -dain
>>
>> On Jun 16, 2008, at 10:56 AM, Dain Sundstrom wrote:
>>
>>> On Jun 13, 2008, at 8:43 AM, jfjames wrote:
>>>
>>>> My intention was to find an alternative to DBCP ready to be used in
>>>> production with OpenEJB. Since it doesn't exist, I have to change
>>>> my mind.
>>>> If you agree to commit our patch quickly, I'm OK to use DBCP in
>>>> production.
>>>
>>> Quick status update. I started working on the patch, but it isn't
>>> as simple as applying it.... When it comes to the DelegateConnection
>>> class, it should appear to the user to be an plain old Connection
>>> without any special pooling properties. This mean that when the
>>> DelegateConnection is closed, it must appear to the caller that the
>>> connection is actually closed, so methods like isClosed should
>>> return true.
>>>
>>> Instead of changing the isClosed method, I'll change the reallyClose
>>> method so it only checks the real connection before closing.
>>>
>>> -dain
>>
>>
>>
>
>
--
View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p18012556.html
Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: DataSource configuration for production
Posted by jfjames <je...@atosorigin.com>.
Thank you Dain for your reactivity. I'm out of my office traveling and having
meetings for two days. A colleague has done some tests which confirms that
your fix is OK. I've planned to confirm by doing my own tests on Thursday.
Dain Sundstrom wrote:
>
> I think I have finally fixed this issue. Can you retest and let me
> know if it is working for you?
>
> Thanks,
>
> -dain
>
> On Jun 16, 2008, at 10:56 AM, Dain Sundstrom wrote:
>
>> On Jun 13, 2008, at 8:43 AM, jfjames wrote:
>>
>>> My intention was to find an alternative to DBCP ready to be used in
>>> production with OpenEJB. Since it doesn't exist, I have to change
>>> my mind.
>>> If you agree to commit our patch quickly, I'm OK to use DBCP in
>>> production.
>>
>> Quick status update. I started working on the patch, but it isn't
>> as simple as applying it.... When it comes to the DelegateConnection
>> class, it should appear to the user to be an plain old Connection
>> without any special pooling properties. This mean that when the
>> DelegateConnection is closed, it must appear to the caller that the
>> connection is actually closed, so methods like isClosed should
>> return true.
>>
>> Instead of changing the isClosed method, I'll change the reallyClose
>> method so it only checks the real connection before closing.
>>
>> -dain
>
>
>
--
View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17913044.html
Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: DataSource configuration for production
Posted by Dain Sundstrom <da...@iq80.com>.
I think I have finally fixed this issue. Can you retest and let me
know if it is working for you?
Thanks,
-dain
On Jun 16, 2008, at 10:56 AM, Dain Sundstrom wrote:
> On Jun 13, 2008, at 8:43 AM, jfjames wrote:
>
>> My intention was to find an alternative to DBCP ready to be used in
>> production with OpenEJB. Since it doesn't exist, I have to change
>> my mind.
>> If you agree to commit our patch quickly, I'm OK to use DBCP in
>> production.
>
> Quick status update. I started working on the patch, but it isn't
> as simple as applying it.... When it comes to the DelegateConnection
> class, it should appear to the user to be an plain old Connection
> without any special pooling properties. This mean that when the
> DelegateConnection is closed, it must appear to the caller that the
> connection is actually closed, so methods like isClosed should
> return true.
>
> Instead of changing the isClosed method, I'll change the reallyClose
> method so it only checks the real connection before closing.
>
> -dain
Re: DataSource configuration for production
Posted by Dain Sundstrom <da...@iq80.com>.
On Jun 13, 2008, at 8:43 AM, jfjames wrote:
> My intention was to find an alternative to DBCP ready to be used in
> production with OpenEJB. Since it doesn't exist, I have to change my
> mind.
> If you agree to commit our patch quickly, I'm OK to use DBCP in
> production.
Quick status update. I started working on the patch, but it isn't as
simple as applying it.... When it comes to the DelegateConnection
class, it should appear to the user to be an plain old Connection
without any special pooling properties. This mean that when the
DelegateConnection is closed, it must appear to the caller that the
connection is actually closed, so methods like isClosed should return
true.
Instead of changing the isClosed method, I'll change the reallyClose
method so it only checks the real connection before closing.
-dain
Re: DataSource configuration for production
Posted by jfjames <je...@atosorigin.com>.
Hi Dain,
In the short-term, I want to focus on production issue.
I've managed to configure C3P0 DataSource as an OpenEJB resource. However, I
don't think it's so interesting as C3P0 is not XA-enabled for the moment and
therefore cannot participate to transactions managed by OpenEJB. I don't
have time enough to spend on C3P0 itself in order to make it XA-enabled.
My intention was to find an alternative to DBCP ready to be used in
production with OpenEJB. Since it doesn't exist, I have to change my mind.
If you agree to commit our patch quickly, I'm OK to use DBCP in production.
I've started implementing an OpenEJB resource which enables to monitor
DataSources through JMX. I've just finished a proof of concept and it seems
to work. Let me know if you think it can be an interesting contribution.
Dain Sundstrom wrote:
>
> If you do get c3p0 (or any other pool) working, can provide the
> configuration so we can add it to the documentation?
>
> One other possibility is atomikos
> (http://www.atomikos.com/Main/AtomikosCommunity
> ). It is a full JTA implementation, so it will be a bit more work to
> integrate.
>
> -dain
>
> On Jun 12, 2008, at 10:07 AM, jfjames wrote:
>
>>
>> In parallel to this DBCP patch, I've developped an OpenEJB service
>> in order
>> to use C3P0. My objective is to have a fallback solution in
>> production in
>> case of problem with DBCP. Everything works fine (very good exercice
>> to
>> understand how OpenEJB configuration works internally ... to be
>> honest I
>> must admit there is still some mystery around XBean in my
>> understanding) ,
>> except that no commit/rollback are sent to the database. This may be
>> due to
>> the fact that C3P0 doesn't support distributed transaction and its
>> connections are not enrolled by the TransactionManager (no
>> implementation of
>> ManagedConnection). There is the same issue with Proxool. Do you
>> know any
>> alternative to DBCP which could be used with OpenEJB ?
>>
>>
>> jfjames wrote:
>>>
>>> OK, we've opened a DBCP JIRA issue :
>>> https://issues.apache.org/jira/browse/DBCP-269.
>>>
>>> It would be very nice to include this patch in OpenEJB 3.0.1.
>>>
>>> -JF
>>>
>>>
>>> David Blevins wrote:
>>>>
>>>>
>>>> On Jun 11, 2008, at 4:44 AM, jfjames wrote:
>>>>
>>>>>
>>>>> We're back ... It seems we’ve identified the cause of the problem.
>>>>> It is
>>>>> located in DBCP 1.3. In fact, the isClosed method of the
>>>>> DelegatingConnection class doesn’t really close the underlying JDBC
>>>>> connection :
>>>>> * when called from the destroyObject method of the
>>>>> PoolableConnectionFactory, the _closed variable is set to false,
>>>>> * therefore the test if ( _closed || _conn.isClosed() ) doesn’t
>>>>> allow to
>>>>> propagate the close along the delegating chain (up to the JDBC
>>>>> connection).
>>>>>
>>>>> According to us, it should be replaced by if ( _closed &&
>>>>> _conn.isClosed()
>>>>> ). We’ve done some tests with this patch and it works fine :
>>>>> maxActive is
>>>>> never exceeded and the number of connections to the database
>>>>> server is
>>>>> stable.
>>>>>
>>>>> Now, we have to check the impact on the DBCP jUnit tests before
>>>>> going
>>>>> further ...
>>>>
>>>> Great work! Going to be a fantastic contribution.
>>>>
>>>> Once you're confident on the impact to the dbcp unit tests, craft
>>>> up a
>>>> patch file and open up a JIRA for the issue at
>>>> http://issues.apache.org/jira/browse/DBCP
>>>> I can help you with instructions on creating patches if you need it
>>>> (just an "svn diff > myPatch.txt" for the most part). Then I can go
>>>> tap those guys on the shoulder and let em know we're waiting on it
>>>> and
>>>> it's critical for us.
>>>>
>>>> If we can get them to commit it we won't need to wait for a DBCP
>>>> release and can just roll up a custom build that we can use in
>>>> 3.0.1.
>>>>
>>>> -David
>>>>
>>>>
>>>>> jfjames wrote:
>>>>>>
>>>>>> We've spent some time today investigating what actually happens in
>>>>>> the
>>>>>> DataSource ... Not so easy since DBCP code is a little tricky !
>>>>>>
>>>>>> We've observed that JDBC connections which are realeased from the
>>>>>> pool by
>>>>>> the Evictor are not physically closed :
>>>>>> 1/ from the DataSource standpoint : the maximum size of the pool
>>>>>> is
>>>>>> never
>>>>>> exceeded (numActive is always inferior to maxActive),
>>>>>> 2/ but from the dataserver standpoint : the number of
>>>>>> connections is
>>>>>> always increasing (up to the maximum allowed by the server).
>>>>>>
>>>>>> We haven't identified the exact cause of this issue : for some
>>>>>> unknown
>>>>>> reason the DelegatingConnection.close() method consider the JDBC
>>>>>> connection as already closed which is wrong.
>>>>>>
>>>>>> Next step tomorrow ...
>>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17775606.html
>>>>> Sent from the OpenEJB User mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17804966.html
>> Sent from the OpenEJB User mailing list archive at Nabble.com.
>>
>
>
>
--
View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17826351.html
Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: DataSource configuration for production
Posted by Dain Sundstrom <da...@iq80.com>.
If you do get c3p0 (or any other pool) working, can provide the
configuration so we can add it to the documentation?
One other possibility is atomikos (http://www.atomikos.com/Main/AtomikosCommunity
). It is a full JTA implementation, so it will be a bit more work to
integrate.
-dain
On Jun 12, 2008, at 10:07 AM, jfjames wrote:
>
> In parallel to this DBCP patch, I've developped an OpenEJB service
> in order
> to use C3P0. My objective is to have a fallback solution in
> production in
> case of problem with DBCP. Everything works fine (very good exercice
> to
> understand how OpenEJB configuration works internally ... to be
> honest I
> must admit there is still some mystery around XBean in my
> understanding) ,
> except that no commit/rollback are sent to the database. This may be
> due to
> the fact that C3P0 doesn't support distributed transaction and its
> connections are not enrolled by the TransactionManager (no
> implementation of
> ManagedConnection). There is the same issue with Proxool. Do you
> know any
> alternative to DBCP which could be used with OpenEJB ?
>
>
> jfjames wrote:
>>
>> OK, we've opened a DBCP JIRA issue :
>> https://issues.apache.org/jira/browse/DBCP-269.
>>
>> It would be very nice to include this patch in OpenEJB 3.0.1.
>>
>> -JF
>>
>>
>> David Blevins wrote:
>>>
>>>
>>> On Jun 11, 2008, at 4:44 AM, jfjames wrote:
>>>
>>>>
>>>> We're back ... It seems we’ve identified the cause of the problem.
>>>> It is
>>>> located in DBCP 1.3. In fact, the isClosed method of the
>>>> DelegatingConnection class doesn’t really close the underlying JDBC
>>>> connection :
>>>> * when called from the destroyObject method of the
>>>> PoolableConnectionFactory, the _closed variable is set to false,
>>>> * therefore the test if ( _closed || _conn.isClosed() ) doesn’t
>>>> allow to
>>>> propagate the close along the delegating chain (up to the JDBC
>>>> connection).
>>>>
>>>> According to us, it should be replaced by if ( _closed &&
>>>> _conn.isClosed()
>>>> ). We’ve done some tests with this patch and it works fine :
>>>> maxActive is
>>>> never exceeded and the number of connections to the database
>>>> server is
>>>> stable.
>>>>
>>>> Now, we have to check the impact on the DBCP jUnit tests before
>>>> going
>>>> further ...
>>>
>>> Great work! Going to be a fantastic contribution.
>>>
>>> Once you're confident on the impact to the dbcp unit tests, craft
>>> up a
>>> patch file and open up a JIRA for the issue at
>>> http://issues.apache.org/jira/browse/DBCP
>>> I can help you with instructions on creating patches if you need it
>>> (just an "svn diff > myPatch.txt" for the most part). Then I can go
>>> tap those guys on the shoulder and let em know we're waiting on it
>>> and
>>> it's critical for us.
>>>
>>> If we can get them to commit it we won't need to wait for a DBCP
>>> release and can just roll up a custom build that we can use in
>>> 3.0.1.
>>>
>>> -David
>>>
>>>
>>>> jfjames wrote:
>>>>>
>>>>> We've spent some time today investigating what actually happens in
>>>>> the
>>>>> DataSource ... Not so easy since DBCP code is a little tricky !
>>>>>
>>>>> We've observed that JDBC connections which are realeased from the
>>>>> pool by
>>>>> the Evictor are not physically closed :
>>>>> 1/ from the DataSource standpoint : the maximum size of the pool
>>>>> is
>>>>> never
>>>>> exceeded (numActive is always inferior to maxActive),
>>>>> 2/ but from the dataserver standpoint : the number of
>>>>> connections is
>>>>> always increasing (up to the maximum allowed by the server).
>>>>>
>>>>> We haven't identified the exact cause of this issue : for some
>>>>> unknown
>>>>> reason the DelegatingConnection.close() method consider the JDBC
>>>>> connection as already closed which is wrong.
>>>>>
>>>>> Next step tomorrow ...
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17775606.html
>>>> Sent from the OpenEJB User mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17804966.html
> Sent from the OpenEJB User mailing list archive at Nabble.com.
>
Re: DataSource configuration for production
Posted by jfjames <je...@atosorigin.com>.
In parallel to this DBCP patch, I've developped an OpenEJB service in order
to use C3P0. My objective is to have a fallback solution in production in
case of problem with DBCP. Everything works fine (very good exercice to
understand how OpenEJB configuration works internally ... to be honest I
must admit there is still some mystery around XBean in my understanding) ,
except that no commit/rollback are sent to the database. This may be due to
the fact that C3P0 doesn't support distributed transaction and its
connections are not enrolled by the TransactionManager (no implementation of
ManagedConnection). There is the same issue with Proxool. Do you know any
alternative to DBCP which could be used with OpenEJB ?
jfjames wrote:
>
> OK, we've opened a DBCP JIRA issue :
> https://issues.apache.org/jira/browse/DBCP-269.
>
> It would be very nice to include this patch in OpenEJB 3.0.1.
>
> -JF
>
>
> David Blevins wrote:
>>
>>
>> On Jun 11, 2008, at 4:44 AM, jfjames wrote:
>>
>>>
>>> We're back ... It seems we’ve identified the cause of the problem.
>>> It is
>>> located in DBCP 1.3. In fact, the isClosed method of the
>>> DelegatingConnection class doesn’t really close the underlying JDBC
>>> connection :
>>> * when called from the destroyObject method of the
>>> PoolableConnectionFactory, the _closed variable is set to false,
>>> * therefore the test if ( _closed || _conn.isClosed() ) doesn’t
>>> allow to
>>> propagate the close along the delegating chain (up to the JDBC
>>> connection).
>>>
>>> According to us, it should be replaced by if ( _closed &&
>>> _conn.isClosed()
>>> ). We’ve done some tests with this patch and it works fine :
>>> maxActive is
>>> never exceeded and the number of connections to the database server is
>>> stable.
>>>
>>> Now, we have to check the impact on the DBCP jUnit tests before going
>>> further ...
>>
>> Great work! Going to be a fantastic contribution.
>>
>> Once you're confident on the impact to the dbcp unit tests, craft up a
>> patch file and open up a JIRA for the issue at
>> http://issues.apache.org/jira/browse/DBCP
>> I can help you with instructions on creating patches if you need it
>> (just an "svn diff > myPatch.txt" for the most part). Then I can go
>> tap those guys on the shoulder and let em know we're waiting on it and
>> it's critical for us.
>>
>> If we can get them to commit it we won't need to wait for a DBCP
>> release and can just roll up a custom build that we can use in 3.0.1.
>>
>> -David
>>
>>
>>> jfjames wrote:
>>>>
>>>> We've spent some time today investigating what actually happens in
>>>> the
>>>> DataSource ... Not so easy since DBCP code is a little tricky !
>>>>
>>>> We've observed that JDBC connections which are realeased from the
>>>> pool by
>>>> the Evictor are not physically closed :
>>>> 1/ from the DataSource standpoint : the maximum size of the pool is
>>>> never
>>>> exceeded (numActive is always inferior to maxActive),
>>>> 2/ but from the dataserver standpoint : the number of connections is
>>>> always increasing (up to the maximum allowed by the server).
>>>>
>>>> We haven't identified the exact cause of this issue : for some
>>>> unknown
>>>> reason the DelegatingConnection.close() method consider the JDBC
>>>> connection as already closed which is wrong.
>>>>
>>>> Next step tomorrow ...
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17775606.html
>>> Sent from the OpenEJB User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>
>
--
View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17804966.html
Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: DataSource configuration for production
Posted by jfjames <je...@atosorigin.com>.
OK, we've opened a DBCP JIRA issue :
https://issues.apache.org/jira/browse/DBCP-269.
It would be very nice to include this patch in OpenEJB 3.0.1.
-JF
David Blevins wrote:
>
>
> On Jun 11, 2008, at 4:44 AM, jfjames wrote:
>
>>
>> We're back ... It seems we’ve identified the cause of the problem.
>> It is
>> located in DBCP 1.3. In fact, the isClosed method of the
>> DelegatingConnection class doesn’t really close the underlying JDBC
>> connection :
>> * when called from the destroyObject method of the
>> PoolableConnectionFactory, the _closed variable is set to false,
>> * therefore the test if ( _closed || _conn.isClosed() ) doesn’t
>> allow to
>> propagate the close along the delegating chain (up to the JDBC
>> connection).
>>
>> According to us, it should be replaced by if ( _closed &&
>> _conn.isClosed()
>> ). We’ve done some tests with this patch and it works fine :
>> maxActive is
>> never exceeded and the number of connections to the database server is
>> stable.
>>
>> Now, we have to check the impact on the DBCP jUnit tests before going
>> further ...
>
> Great work! Going to be a fantastic contribution.
>
> Once you're confident on the impact to the dbcp unit tests, craft up a
> patch file and open up a JIRA for the issue at
> http://issues.apache.org/jira/browse/DBCP
> I can help you with instructions on creating patches if you need it
> (just an "svn diff > myPatch.txt" for the most part). Then I can go
> tap those guys on the shoulder and let em know we're waiting on it and
> it's critical for us.
>
> If we can get them to commit it we won't need to wait for a DBCP
> release and can just roll up a custom build that we can use in 3.0.1.
>
> -David
>
>
>> jfjames wrote:
>>>
>>> We've spent some time today investigating what actually happens in
>>> the
>>> DataSource ... Not so easy since DBCP code is a little tricky !
>>>
>>> We've observed that JDBC connections which are realeased from the
>>> pool by
>>> the Evictor are not physically closed :
>>> 1/ from the DataSource standpoint : the maximum size of the pool is
>>> never
>>> exceeded (numActive is always inferior to maxActive),
>>> 2/ but from the dataserver standpoint : the number of connections is
>>> always increasing (up to the maximum allowed by the server).
>>>
>>> We haven't identified the exact cause of this issue : for some
>>> unknown
>>> reason the DelegatingConnection.close() method consider the JDBC
>>> connection as already closed which is wrong.
>>>
>>> Next step tomorrow ...
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17775606.html
>> Sent from the OpenEJB User mailing list archive at Nabble.com.
>>
>>
>
>
>
--
View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17796021.html
Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: DataSource configuration for production
Posted by David Blevins <da...@visi.com>.
On Jun 11, 2008, at 4:44 AM, jfjames wrote:
>
> We're back ... It seems we’ve identified the cause of the problem.
> It is
> located in DBCP 1.3. In fact, the isClosed method of the
> DelegatingConnection class doesn’t really close the underlying JDBC
> connection :
> * when called from the destroyObject method of the
> PoolableConnectionFactory, the _closed variable is set to false,
> * therefore the test if ( _closed || _conn.isClosed() ) doesn’t
> allow to
> propagate the close along the delegating chain (up to the JDBC
> connection).
>
> According to us, it should be replaced by if ( _closed &&
> _conn.isClosed()
> ). We’ve done some tests with this patch and it works fine :
> maxActive is
> never exceeded and the number of connections to the database server is
> stable.
>
> Now, we have to check the impact on the DBCP jUnit tests before going
> further ...
Great work! Going to be a fantastic contribution.
Once you're confident on the impact to the dbcp unit tests, craft up a
patch file and open up a JIRA for the issue at http://issues.apache.org/jira/browse/DBCP
I can help you with instructions on creating patches if you need it
(just an "svn diff > myPatch.txt" for the most part). Then I can go
tap those guys on the shoulder and let em know we're waiting on it and
it's critical for us.
If we can get them to commit it we won't need to wait for a DBCP
release and can just roll up a custom build that we can use in 3.0.1.
-David
> jfjames wrote:
>>
>> We've spent some time today investigating what actually happens in
>> the
>> DataSource ... Not so easy since DBCP code is a little tricky !
>>
>> We've observed that JDBC connections which are realeased from the
>> pool by
>> the Evictor are not physically closed :
>> 1/ from the DataSource standpoint : the maximum size of the pool is
>> never
>> exceeded (numActive is always inferior to maxActive),
>> 2/ but from the dataserver standpoint : the number of connections is
>> always increasing (up to the maximum allowed by the server).
>>
>> We haven't identified the exact cause of this issue : for some
>> unknown
>> reason the DelegatingConnection.close() method consider the JDBC
>> connection as already closed which is wrong.
>>
>> Next step tomorrow ...
>>
>
> --
> View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17775606.html
> Sent from the OpenEJB User mailing list archive at Nabble.com.
>
>
Re: DataSource configuration for production
Posted by jfjames <je...@atosorigin.com>.
We're back ... It seems we’ve identified the cause of the problem. It is
located in DBCP 1.3. In fact, the isClosed method of the
DelegatingConnection class doesn’t really close the underlying JDBC
connection :
* when called from the destroyObject method of the
PoolableConnectionFactory, the _closed variable is set to false,
* therefore the test if ( _closed || _conn.isClosed() ) doesn’t allow to
propagate the close along the delegating chain (up to the JDBC connection).
According to us, it should be replaced by if ( _closed && _conn.isClosed()
). We’ve done some tests with this patch and it works fine : maxActive is
never exceeded and the number of connections to the database server is
stable.
Now, we have to check the impact on the DBCP jUnit tests before going
further ...
jfjames wrote:
>
> We've spent some time today investigating what actually happens in the
> DataSource ... Not so easy since DBCP code is a little tricky !
>
> We've observed that JDBC connections which are realeased from the pool by
> the Evictor are not physically closed :
> 1/ from the DataSource standpoint : the maximum size of the pool is never
> exceeded (numActive is always inferior to maxActive),
> 2/ but from the dataserver standpoint : the number of connections is
> always increasing (up to the maximum allowed by the server).
>
> We haven't identified the exact cause of this issue : for some unknown
> reason the DelegatingConnection.close() method consider the JDBC
> connection as already closed which is wrong.
>
> Next step tomorrow ...
>
--
View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17775606.html
Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: DataSource configuration for production
Posted by jfjames <je...@atosorigin.com>.
We've spent some time today investigating what actually happens in the
DataSource ... Not so easy since DBCP code is a little tricky !
We've observed that JDBC connections which are realeased from the pool by
the Evictor are not physically closed :
1/ from the DataSource standpoint : the maximum size of the pool is never
exceeded (numActive is always inferior to maxActive),
2/ but from the dataserver standpoint : the number of connections is always
increasing (up to the maximum allowed by the server).
We haven't identified the exact cause of this issue : for some unknown
reason the DelegatingConnection.close() method consider the JDBC connection
as already closed which is wrong.
Next step tomorrow ...
--
View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17760106.html
Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: DataSource configuration for production
Posted by David Blevins <da...@visi.com>.
On Jun 10, 2008, at 12:49 AM, jfjames wrote:
> Sorry David, but the MAVEN source jar is empty ...
Looks like the maven-shade-plugin needs some bug fixing :)
Oh well. The two svn commands I post should give all the required
source.
-David
Re: DataSource configuration for production
Posted by jfjames <je...@atosorigin.com>.
Sorry David, but the MAVEN source jar is empty ...
--
View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17749654.html
Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: DataSource configuration for production
Posted by David Blevins <da...@visi.com>.
On Jun 9, 2008, at 3:22 AM, jfjames wrote:
> We're going to spend some times this afternoon investigating what
> really
> happens in DBCP and COMMON-POOLS. BTW do you know how we can access
> the JAVA
> source of DBCP and COMMON-POOLS embedded in OPENJB 3.0 ?
I'll see if I can give your scenario a try as well. (trying
desperately to finish up some cmp stuff)
Here's the source for both those modules:
http://people.apache.org/repo/m2-ibiblio-rsync-repository/org/apache/openejb/commons-dbcp-all/1.3/commons-dbcp-all-1.3-sources.jar
That's the maven created one, which may not include build files, etc.
Here's how to get it straight from svn:
svn co -r617003 http://svn.apache.org/repos/asf/commons/proper/dbcp/trunk
commons-dbcp
svn http://svn.apache.org/repos/asf/commons/proper/pool/tags/POOL_1_3
commons-pool
-David
Re: DataSource configuration for production
Posted by jfjames <je...@atosorigin.com>.
Hi David,
I suggest the following test which demonstrates that maxActive is not
checked properly without a line of Java code :
1/ Define a Datasource with an eviction policy configured :
<Resource id="jjsDS" type="DataSource">
JdbcDriver com.mysql.jdbc.Driver
JdbcUrl jdbc:mysql://localhost:3306/jjs?cacheResultsetMetadata=true
UserName xxxx
Password yyyy
JtaManaged true
InitialSize 5
MaxActive 10
MaxWait -1
ValidationQuery SELECT 1
TestOnBorrow false
TestOnReturn false
PoolPreparedStatements true
MaxOpenPreparedStatements 10
TimeBetweenEvictionRunsMillis 10000
MinEvictableIdleTimeMillis 20000
NumTestsPerEvictionRun 3
TestWhileIdle false
MinIdle 5
MaxIdle 10
</Resource>
2/ Reference it from the persistence.xml of your application :
<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
version="1.0">
<persistence-unit name="manager_bench" transaction-type="JTA">
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<jta-data-source>java:openejb/Resource/jjsDS</jta-data-source>
<non-jta-data-source>java:openejb/Resource/jjsUnmanagedDS</non-jta-data-source>
<!--
<mapping-file>META-INF/orm.xml</mapping-file>
-->
<properties>
<property name="hibernate.dialect"
value="org.hibernate.dialect.MySQLInnoDBDialect" />
<!-- property name="hibernate.hbm2ddl.auto" value="create-drop" -->
<property name="hibernate.show_sql" value="false" />
<property name="hibernate.transaction.manager_lookup_class"
value="org.apache.openejb.hibernate.TransactionManagerLookup" />
<property name="hibernate.max_fetch_depth" value="2" />
<property name="hibernate.jdbc.batch_size" value="0" />
<property name="hibernate.cache.provider_class"
value="org.hibernate.cache.EhCacheProvider" />
<property name="hibernate.cache.use_query_cache"
value="true" />
<property name="hibernate.connection.release_mode"
value="auto" />
</properties>
</persistence-unit>
</persistence>
In that context, after the application deployment, you can see (with MySQL
Administrator for instance) that the number of connections to the database
never stops growing. Each time the evictor runs, a new slot of three 3
connections is opened (value of NumTestsPerEvictionRun). I've run the same
test againsts HSQLDB : the result is the same. A colleague has also
reproduced it.
We're going to spend some times this afternoon investigating what really
happens in DBCP and COMMON-POOLS. BTW do you know how we can access the JAVA
source of DBCP and COMMON-POOLS embedded in OPENJB 3.0 ?
Thanks in advance.
--
View this message in context: http://www.nabble.com/DataSource-configuration-for-production-tp17695975p17730022.html
Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: DataSource configuration for production
Posted by David Blevins <da...@visi.com>.
On Jun 6, 2008, at 9:10 AM, jfjames wrote:
>
> I'm doing some tests before putting OpenEJB in production and I'm
> facing a
> problem with the DataSource configuration.
> My environment is :
> 1/ Application deployed on Tomcat 6.0.16 + Open EJB 3.0,
> 2/ Database : MySQL 4.0.1 + Mysql Connector 5.0.4
> 3/ JPA provider : Hibernate Entity Manager 3.3.1 (with
> hibernate.connection.release_mode parameter set to auto)
> 4/ Sun JVM 1.5.0 on Windows NT (Linux in production)
> 5/ DataSource configuration : no Evictor configured
> (TimeBetweenEvictionRunsMillis=-1), MaxActive=10 and MaxWait=-1.
> My application exposes a JAX-WS interface and I use a benchmark
> based on a
> multi-threaded client injector.
> My problem is : when the number of concurrent injector threads
> exceeds the
> maxActive parameter of the DataSource, many database connections are
> opened.
> For instance more than 50 connexions for 10 running threads ... far
> more
> than MaxActive !
> I've checked the DBCP and COMMON POOLS code source and normally this
> should
> not happen : when no connection is available from the pool, we
> should wait
> without creating new one (MaxWait=-1). So why are new connections
> created in
> such quantity ?
> Has anyone encountered and resolve this kind of problems ? I need to
> resolve
> it before going to production.
> Any help appreciated.
Is it possible you could create a small app + test case to reproduce
it? I suspect we'll need it in order to figure out what might be
happening. Doesn't have to be pretty, can even submit the code you're
using now if you're ok with that.
-David