You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jdo-dev@db.apache.org by Michael Watzek <mw...@spree.de> on 2005/06/02 17:32:12 UTC
JIRA JDO-53
Hi Andy,
JIRA JDO-53 suggests to use JPOX connection pooling to improve TCK
performance.
I followed the instructions decribed in
"http://www.jpox.org/docs/1_1/rdbms_connection_pooling.html" and ran
JPOX 1.1.0-beta-3 with commons-dbcp. To me, it looks like there is no
performance improvement executing the TCK.
So maybe, I made a mistake and you can help. This is what I have done:
1) Downloaded JPOX plugin for commons-dbcp (jpox-dbcp-1.1.0-beta-3.jar)
2) Downloaded commons-dbcp (commons-dbcp-1.2.1.jar)
3) Downloaded commons-pool (commons-pool-1.2.jar)
4) Downloaded commons-collections (commons-collections-3.1.jar)
5) Added a JPOX property to PMF properties
(org.jpox.connectionPoolingType=DBCP)
6) Added all jars to the classpath and started the TCK
Is there anything missing?
Please note that each TCK test has the following pattern:
pmf = JDOHelper.getPersistenceManagerFactory(..);
...
pmf.close();
Does PMF.close invalidate or clear the connection pool?
Regards,
Michael
--
-------------------------------------------------------------------
Michael Watzek Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de Buelowstr. 66
Tel.: ++49/30/235 520 36 10783 Berlin - Germany
Fax.: ++49/30/217 520 12 http://www.spree.de/
-------------------------------------------------------------------
Re: JIRA JDO-53
Posted by Andy Jefferson <an...@jpox.org>.
> The JPOX log file does neither contain the strings "Datastore Connection
> Pooling" nor "DBCP". Do I have to run JPOX on a particular log level? To
> me, it looks like we run JPOX on "DEBUG".
You need the log4j category "log4j.category.JPOX.JDO" set at DEBUG level to
see these.
> I have tried to set the JPOX property "org.jpox.connectionPoolingType"
> to "None". I get the exception below.
That is fixed in CVS, but is not important since you could just omit the
setting if you don't want connection pooling.
> I also tried to exclude commons-dbcp from the classpath. I got a
> ClassNotFoundException. So I think, that connection pooling is used.
That is a reasonable assumption. The log would confirm it - and DBCP probably
has log categories that can be enabled to see the gory details of its pooling
process if you really need to know.
Always gives significant speed ups for me. Really depends on the test and
whether you do more than 1 or 2 txns or DB calls in a test.
--
Andy
JPOX - Java Persistent Objects
Re: JIRA JDO-53
Posted by Michael Watzek <mw...@spree.de>.
Hi Andy,
>>This is what I have done:
>>
>>1) Downloaded JPOX plugin for commons-dbcp (jpox-dbcp-1.1.0-beta-3.jar)
>>2) Downloaded commons-dbcp (commons-dbcp-1.2.1.jar)
>>3) Downloaded commons-pool (commons-pool-1.2.jar)
>>4) Downloaded commons-collections (commons-collections-3.1.jar)
>>5) Added a JPOX property to PMF properties
>>(org.jpox.connectionPoolingType=DBCP)
>>6) Added all jars to the classpath and started the TCK
>>
>>Is there anything missing?
>
>
> Sounds about right. Have you looked in the log to see if it has recognised
> your connection pool setting ? (there is a block of messages printed when you
> open the PMF and it should say something like "Datastore Connection Pooling :
> DBCP")
The JPOX log file does neither contain the strings "Datastore Connection
Pooling" nor "DBCP". Do I have to run JPOX on a particular log level? To
me, it looks like we run JPOX on "DEBUG".
I have tried to set the JPOX property "org.jpox.connectionPoolingType"
to "None". I get the exception below. I do not get this exception when I
set the property to "DBCP" or "C3P0". I also tried to exclude
commons-dbcp from the classpath. I got a ClassNotFoundException. So I
think, that connection pooling is used.
Regards,
Michael
javax.jdo.JDOUserException: Invalid Connection Pooling type. Please
check your specified type against the supported types in the JPOX
documentation, please specify the DataSource directly
at
org.jpox.PMFConfiguration.setConnectionPoolingType(PMFConfiguration.java:962)
at
org.jpox.PersistenceManagerFactoryImpl$37.set(PersistenceManagerFactoryImpl.java:362)
at
org.jpox.PersistenceManagerFactoryImpl.setOptions(PersistenceManagerFactoryImpl.java:619)
at
org.jpox.PersistenceManagerFactoryImpl.getPersistenceManagerFactory(PersistenceManagerFactoryImpl.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:299)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:254)
at org.apache.jdo.tck.JDO_Test.getPMF(JDO_Test.java:386)
at org.apache.jdo.tck.JDO_Test.setUp(JDO_Test.java:163)
at org.apache.jdo.tck.JDO_Test.runBare(JDO_Test.java:180)
at org.apache.jdo.tck.util.BatchTestRunner.run(BatchTestRunner.java:75)
at org.apache.jdo.tck.util.BatchTestRunner.run(BatchTestRunner.java:70)
at
org.apache.jdo.tck.api.persistencemanagerfactory.AfterCloseSetMethodsThrowException.main(AfterCloseSetMethodsThrowException.java:62)
--
-------------------------------------------------------------------
Michael Watzek Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de Buelowstr. 66
Tel.: ++49/30/235 520 36 10783 Berlin - Germany
Fax.: ++49/30/217 520 12 http://www.spree.de/
-------------------------------------------------------------------
Re: Patch for JDO-53
Posted by Michael Watzek <mw...@spree.de>.
Hi Michelle,
> Hi Michael,
>
> Version 0.8.5.2 of c3p0 is not available on ibiblio.org. Version
> 0.8.4.5 is available. Is it possible to use the earlier version? (It
> looks like I get some additional errors on the TCK using the earlier
> version.)
I can not recommend using version 0.8.5.2. I got lots of errors
(something like 300).
If not, can you provide a URL for downloading the later version?
There is a new prerelease 6 available for download. This produces
reasonable amount of errors/failures. I got the same amount of errors
without connection pooling:
Datastore identity:
Tests run: 350, Failures: 13, Errors: 88 (Time: 625 seconds)
Application identity:
Tests run: 350, Failures: 12, Errors: 84 (Time: 768 seconds)
The download URL is:
http://prdownloads.sourceforge.net/c3p0/c3p0-0.9.0-pre6.bin.zip?use_mirror=switch
Attached you find a patch for the project.xml, containing that version
together with the doanload URL. "jdori.properties" are set to use c3p0
connection pooling with this patch.
Regards,
Michael
>
> Thanks,
> Michelle
>
> Michael Watzek wrote:
>
>> Hi,
>>
>> In order to run you need to download the following jars and store them
>> in your local maven repository:
>>
>> 1) jpox-dbcp-1.1.0-beta-3.jar to jpox/jars
>> 2) jpox-c3p0-1.1.0-beta-3.jar to jpox/jars
>> 3) commons-dbcp-1.2.1.jar to commons-dbcp/jars
>> 4) c3p0-0.8.5.2.jar to c3p0/jars
>> 5) commons-pool-1.2.jar to commons-pool/jars
>> 6) commons-collections-3.1.jar to commons-collections/jars
>>
>> Regards,
>> Michael
>>
--
-------------------------------------------------------------------
Michael Watzek Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de Buelowstr. 66
Tel.: ++49/30/235 520 36 10783 Berlin - Germany
Fax.: ++49/30/217 520 12 http://www.spree.de/
-------------------------------------------------------------------
Re: Patch for JDO-53
Posted by Michelle Caisse <Mi...@Sun.COM>.
Hi Michael,
Version 0.8.5.2 of c3p0 is not available on ibiblio.org. Version
0.8.4.5 is available. Is it possible to use the earlier version? (It
looks like I get some additional errors on the TCK using the earlier
version.) If not, can you provide a URL for downloading the later version?
Thanks,
Michelle
Michael Watzek wrote:
> Hi,
>
> In order to run you need to download the following jars and store them
> in your local maven repository:
>
> 1) jpox-dbcp-1.1.0-beta-3.jar to jpox/jars
> 2) jpox-c3p0-1.1.0-beta-3.jar to jpox/jars
> 3) commons-dbcp-1.2.1.jar to commons-dbcp/jars
> 4) c3p0-0.8.5.2.jar to c3p0/jars
> 5) commons-pool-1.2.jar to commons-pool/jars
> 6) commons-collections-3.1.jar to commons-collections/jars
>
> Regards,
> Michael
>
Patch for JDO-53
Posted by Michael Watzek <mw...@spree.de>.
Hi,
I'm not sure how to continue on this issue. I see three options:
1) Check in the changes though they do not have the expected effect.
2) Leave the code unchanged and close the bug.
3) Check in the changes and change the coding pattern for TCK tests, so
that all tests share the same PMF instance.
I vote for option 1). Below I listed the changes for option 1):
1) Additional dependencies to JPOX plugins, DBCP, C3P0, commons-pool,
commons-collections.
2) Change in propject.properties, maven.xml to extend the TCK classpath
3) Add a JPOX property to PMF properties
I have added a patch file containing these changes. The connection pool
in this patch is set to DBCP. You can switch to C3P0 if you set JPOX
property "org.jpox.connectionPoolingType" to "C3P0".
In order to run you need to download the following jars and store them
in your local maven repository:
1) jpox-dbcp-1.1.0-beta-3.jar to jpox/jars
2) jpox-c3p0-1.1.0-beta-3.jar to jpox/jars
3) commons-dbcp-1.2.1.jar to commons-dbcp/jars
4) c3p0-0.8.5.2.jar to c3p0/jars
5) commons-pool-1.2.jar to commons-pool/jars
6) commons-collections-3.1.jar to commons-collections/jars
Regards,
Michael
>>> I'm just curious if two created PMF instances share the same connection
>>> pool?
>>
>>
>>
>> No. You create a PMF and it gets its own (DBCP/C3P0/...) DataSource,
>> and hence it gets its own connection pool (unless DBCP/C3P0 are
>> intelligent enough to realise when 2 PMF connection URLs and login
>> credentials are the same and so share with the already existing data
>> source ...)
>
> This is very likely the reason why I do not see performance improvement
> in TCK runs: Many tests execute only 1 transaction and each test uses a
> separate PMF instance.
>
> Thanks,
> Michael
--
-------------------------------------------------------------------
Michael Watzek Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de Buelowstr. 66
Tel.: ++49/30/235 520 36 10783 Berlin - Germany
Fax.: ++49/30/217 520 12 http://www.spree.de/
-------------------------------------------------------------------
Re: JIRA JDO-53
Posted by Michael Watzek <mw...@spree.de>.
Hi Andy,
>>>In your case JPOX will be creating a DBCP DataSource, instead of the
>>>normal DataSource, and in PMF.close() this is nulled out. Looking at the
>>>code it isn't explicitly closed - I'll have to check if the DBCP class
>>>provides a close() method.
>>
>>I'm just curious if two created PMF instances share the same connection
>>pool?
>
>
> No.
> You create a PMF and it gets its own (DBCP/C3P0/...) DataSource, and hence it
> gets its own connection pool (unless DBCP/C3P0 are intelligent enough to
> realise when 2 PMF connection URLs and login credentials are the same and so
> share with the already existing data source ...)
This is very likely the reason why I do not see performance improvement
in TCK runs: Many tests execute only 1 transaction and each test uses a
separate PMF instance.
Thanks,
Michael
--
-------------------------------------------------------------------
Michael Watzek Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de Buelowstr. 66
Tel.: ++49/30/235 520 36 10783 Berlin - Germany
Fax.: ++49/30/217 520 12 http://www.spree.de/
-------------------------------------------------------------------
Re: JIRA JDO-53
Posted by Craig Russell <Cr...@Sun.COM>.
I would be very surprised to find any connection pooling done between a
PMF constructed in one test and a PMF constructed in the next test. We
explicitly close the PMF after a test is complete.
Craig
On Jun 2, 2005, at 9:01 AM, Andy Jefferson wrote:
>>> In your case JPOX will be creating a DBCP DataSource, instead of the
>>> normal DataSource, and in PMF.close() this is nulled out. Looking at
>>> the
>>> code it isn't explicitly closed - I'll have to check if the DBCP
>>> class
>>> provides a close() method.
>>
>> I'm just curious if two created PMF instances share the same
>> connection
>> pool?
>
> No.
> You create a PMF and it gets its own (DBCP/C3P0/...) DataSource, and
> hence it
> gets its own connection pool (unless DBCP/C3P0 are intelligent enough
> to
> realise when 2 PMF connection URLs and login credentials are the same
> and so
> share with the already existing data source ...)
>
> --
> Andy
> JPOX - Java Persistent Objects
>
Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!
Re: JIRA JDO-53
Posted by Andy Jefferson <an...@jpox.org>.
> > In your case JPOX will be creating a DBCP DataSource, instead of the
> > normal DataSource, and in PMF.close() this is nulled out. Looking at the
> > code it isn't explicitly closed - I'll have to check if the DBCP class
> > provides a close() method.
>
> I'm just curious if two created PMF instances share the same connection
> pool?
No.
You create a PMF and it gets its own (DBCP/C3P0/...) DataSource, and hence it
gets its own connection pool (unless DBCP/C3P0 are intelligent enough to
realise when 2 PMF connection URLs and login credentials are the same and so
share with the already existing data source ...)
--
Andy
JPOX - Java Persistent Objects
Re: JIRA JDO-53
Posted by Michael Watzek <mw...@spree.de>.
Hi Andy,
> In your case JPOX will be creating a DBCP DataSource, instead of the normal
> DataSource, and in PMF.close() this is nulled out. Looking at the code it
> isn't explicitly closed - I'll have to check if the DBCP class provides a
> close() method.
I'm just curious if two created PMF instances share the same connection
pool?
Regards,
Michael
--
-------------------------------------------------------------------
Michael Watzek Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de Buelowstr. 66
Tel.: ++49/30/235 520 36 10783 Berlin - Germany
Fax.: ++49/30/217 520 12 http://www.spree.de/
-------------------------------------------------------------------
Re: JIRA JDO-53
Posted by Andy Jefferson <an...@jpox.org>.
Hi Michael,
> This is what I have done:
>
> 1) Downloaded JPOX plugin for commons-dbcp (jpox-dbcp-1.1.0-beta-3.jar)
> 2) Downloaded commons-dbcp (commons-dbcp-1.2.1.jar)
> 3) Downloaded commons-pool (commons-pool-1.2.jar)
> 4) Downloaded commons-collections (commons-collections-3.1.jar)
> 5) Added a JPOX property to PMF properties
> (org.jpox.connectionPoolingType=DBCP)
> 6) Added all jars to the classpath and started the TCK
>
> Is there anything missing?
Sounds about right. Have you looked in the log to see if it has recognised
your connection pool setting ? (there is a block of messages printed when you
open the PMF and it should say something like "Datastore Connection Pooling :
DBCP")
> Please note that each TCK test has the following pattern:
>
> pmf = JDOHelper.getPersistenceManagerFactory(..);
> ...
> pmf.close();
>
> Does PMF.close invalidate or clear the connection pool?
In your case JPOX will be creating a DBCP DataSource, instead of the normal
DataSource, and in PMF.close() this is nulled out. Looking at the code it
isn't explicitly closed - I'll have to check if the DBCP class provides a
close() method.
--
Andy
JPOX - Java Persistent Objects