You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openjpa.apache.org by "Kevin Sutter (JIRA)" <ji...@apache.org> on 2007/01/29 20:21:49 UTC
[jira] Created: (OPENJPA-115) Bottleneck(s) with using OpenJPA in a
Container-managed environment
Bottleneck(s) with using OpenJPA in a Container-managed environment
-------------------------------------------------------------------
Key: OPENJPA-115
URL: https://issues.apache.org/jira/browse/OPENJPA-115
Project: OpenJPA
Issue Type: Bug
Components: kernel
Reporter: Kevin Sutter
Assigned To: Kevin Sutter
Priority: Critical
Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
Kevin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OPENJPA-115) Bottleneck(s) with using OpenJPA in
a Container-managed environment
Posted by "Patrick Linskey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12468709 ]
Patrick Linskey commented on OPENJPA-115:
-----------------------------------------
> Our original experimentation showed that even with the weak references
> in ReferenceHashSet, we still bogged down during the cleanup of the
> _brokers collection.
But that should only be happening when the EMF is closed, right? Presumably, that corresponds to appserver shutdown / application undeploy. Is your test measuring that part of the application lifecycle?
> Bottleneck(s) with using OpenJPA in a Container-managed environment
> -------------------------------------------------------------------
>
> Key: OPENJPA-115
> URL: https://issues.apache.org/jira/browse/OPENJPA-115
> Project: OpenJPA
> Issue Type: Bug
> Components: kernel
> Reporter: Kevin Sutter
> Assigned To: Kevin Sutter
> Priority: Critical
>
> Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
> Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
> Kevin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OPENJPA-115) Bottleneck(s) with using OpenJPA in
a Container-managed environment
Posted by "Patrick Linskey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12468397 ]
Patrick Linskey commented on OPENJPA-115:
-----------------------------------------
>From what I can tell, the locking is needed to guard the _brokers collection. (It is also guarding the _transactional collection, but findExisting is always false in the OpenJPA codebase, so that code is never used in OpenJPA.) In IMO, the guarding is overly-granular at best, and unneeded at worst.
The _brokers collection provides us with a means to ensure that when we close a BrokerFactory, all open Brokers are closed along with it. Additionally, we use a weak reference map here, so we don't need to maintain the _brokers collection during Broker.close().
So, we have a number of options:
1. move the locking logic to be just around the code that maintains the _brokers collection
2. implement a concurrent Collection class that uses weak references, and eliminate all locking
3. create a configuration option (openjpa.CloseBrokersOnBrokerFactoryClose or something) that controls whether or not OpenJPA tracks open brokers. This would allow default behavior to be pro-active about cleanup, but also allow containers (which are presumably doing a good job of resource clean-up) to bypass the overhead.
> Bottleneck(s) with using OpenJPA in a Container-managed environment
> -------------------------------------------------------------------
>
> Key: OPENJPA-115
> URL: https://issues.apache.org/jira/browse/OPENJPA-115
> Project: OpenJPA
> Issue Type: Bug
> Components: kernel
> Reporter: Kevin Sutter
> Assigned To: Kevin Sutter
> Priority: Critical
>
> Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
> Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
> Kevin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (OPENJPA-115) Bottleneck(s) with using OpenJPA in
a Container-managed environment
Posted by "Kevin Sutter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Sutter resolved OPENJPA-115.
----------------------------------
Resolution: Fixed
Changes committed to SVN.
> Bottleneck(s) with using OpenJPA in a Container-managed environment
> -------------------------------------------------------------------
>
> Key: OPENJPA-115
> URL: https://issues.apache.org/jira/browse/OPENJPA-115
> Project: OpenJPA
> Issue Type: Bug
> Components: kernel
> Reporter: Kevin Sutter
> Assigned To: Kevin Sutter
> Priority: Critical
>
> Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
> Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
> Kevin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OPENJPA-115) Bottleneck(s) with using OpenJPA in
a Container-managed environment
Posted by "Kevin Sutter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12468616 ]
Kevin Sutter commented on OPENJPA-115:
--------------------------------------
Thanks, Patrick and Craig. I did some looking in lib and found that we also have a ConcurrentReferenceHashSet. So, I have modified _brokers accordingly and am waiting for our test results. I have also removed the explicit lock/unlock invocations. As Patrick pointed out, the findBroker() is never called since findExisting is always false for OpenJPA, thus the other synch block shouldn't come into play. Hopefully, with the weak references in ConcurrentReferenceHashSet _brokers, this should help with the blocking and broker cleanup. Our original experimentation showed that even with the weak references in ReferenceHashSet, we still bogged down during the cleanup of the _brokers collection. But, maybe there's a problem with SunOne where they are not properly closing the EntityManagers. I'll post back when we get the test results.
> Bottleneck(s) with using OpenJPA in a Container-managed environment
> -------------------------------------------------------------------
>
> Key: OPENJPA-115
> URL: https://issues.apache.org/jira/browse/OPENJPA-115
> Project: OpenJPA
> Issue Type: Bug
> Components: kernel
> Reporter: Kevin Sutter
> Assigned To: Kevin Sutter
> Priority: Critical
>
> Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
> Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
> Kevin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (OPENJPA-115) Bottleneck(s) with using OpenJPA in a
Container-managed environment
Posted by "Patrick Linskey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Linskey updated OPENJPA-115:
------------------------------------
Description:
Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
Kevin
was:
Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
Kevin
The null problem is probably due to references being garbage collected -- this is a typical side-effect of weak reference collection types. The improved concurrency is typically worth the extra difficulty of checking for nulls, though.
> Bottleneck(s) with using OpenJPA in a Container-managed environment
> -------------------------------------------------------------------
>
> Key: OPENJPA-115
> URL: https://issues.apache.org/jira/browse/OPENJPA-115
> Project: OpenJPA
> Issue Type: Bug
> Components: kernel
> Reporter: Kevin Sutter
> Assigned To: Kevin Sutter
> Priority: Critical
>
> Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
> Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
> Kevin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OPENJPA-115) Bottleneck(s) with using OpenJPA in
a Container-managed environment
Posted by "Patrick Linskey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12468429 ]
Patrick Linskey commented on OPENJPA-115:
-----------------------------------------
Recall that we do have a 1.4-compatible concurrent collection at org.apache.openjpa.lib.util.concurrent.ConcurrentHashSet. However, to my knowledge, that impl uses hard references.
> Bottleneck(s) with using OpenJPA in a Container-managed environment
> -------------------------------------------------------------------
>
> Key: OPENJPA-115
> URL: https://issues.apache.org/jira/browse/OPENJPA-115
> Project: OpenJPA
> Issue Type: Bug
> Components: kernel
> Reporter: Kevin Sutter
> Assigned To: Kevin Sutter
> Priority: Critical
>
> Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
> Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
> Kevin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OPENJPA-115) Bottleneck(s) with using OpenJPA in
a Container-managed environment
Posted by "Craig Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12469604 ]
Craig Russell commented on OPENJPA-115:
---------------------------------------
Yes, null checking is an expected requirement when dealing with weak references. If you can get a strong reference in a cleanup routine then it's ok to use it, because the fact that you have a strong reference will prevent garbage collection.
If you get null from your weak reference, then the garbage collector has already done its job and by definition there can be nothing left for your routine to clean up.
> Bottleneck(s) with using OpenJPA in a Container-managed environment
> -------------------------------------------------------------------
>
> Key: OPENJPA-115
> URL: https://issues.apache.org/jira/browse/OPENJPA-115
> Project: OpenJPA
> Issue Type: Bug
> Components: kernel
> Reporter: Kevin Sutter
> Assigned To: Kevin Sutter
> Priority: Critical
>
> Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
> Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
> Kevin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OPENJPA-115) Bottleneck(s) with using OpenJPA in
a Container-managed environment
Posted by "Kevin Sutter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12469598 ]
Kevin Sutter commented on OPENJPA-115:
--------------------------------------
I'm going to go ahead with the proposed changes to remove the explicit lock/unlock invocations, and change the _brokers definition to use ConcurrentReferenceHashSet with weak references. The problem at shutdown was with the EMF.close processing. It is attempting to close any outstanding EM's that had not been closed. But, as it was iterating through this list, one of the EM instances became null and caused an NPE. Not sure if this is due to the weak references being GC'd or it indicates a possible problem with the underlying Collections MapBackedSet or something else, but a simple null check gets around this problem.
With these changes, the initial bottleneck with running OpenJPA in a server environment has been resolved. We are still pursuing other performance concerns, but we'll open new JIRA reports when we get something more concrete.
Kevin
> Bottleneck(s) with using OpenJPA in a Container-managed environment
> -------------------------------------------------------------------
>
> Key: OPENJPA-115
> URL: https://issues.apache.org/jira/browse/OPENJPA-115
> Project: OpenJPA
> Issue Type: Bug
> Components: kernel
> Reporter: Kevin Sutter
> Assigned To: Kevin Sutter
> Priority: Critical
>
> Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
> Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
> Kevin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OPENJPA-115) Bottleneck(s) with using OpenJPA in
a Container-managed environment
Posted by "Craig Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENJPA-115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12468408 ]
Craig Russell commented on OPENJPA-115:
---------------------------------------
Kevin opined: Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers.
I believe that this applies only to injected EntityManagers, not to those created by the user explicitly via EMF.createEntityManager. So there is still some cleanup needed to guarantee proper resource deallocation when the EMF is closed.
Patrick offered: implement a concurrent Collection class that uses weak references, and eliminate all locking
I believe that this is an optimal solution. Concurrent Collections should have almost zero locking in most containers.
> Bottleneck(s) with using OpenJPA in a Container-managed environment
> -------------------------------------------------------------------
>
> Key: OPENJPA-115
> URL: https://issues.apache.org/jira/browse/OPENJPA-115
> Project: OpenJPA
> Issue Type: Bug
> Components: kernel
> Reporter: Kevin Sutter
> Assigned To: Kevin Sutter
> Priority: Critical
>
> Running some benchmarks against OpenJPA using the Sun Java System (SunOne) application server. Under load, we're not able to push the cpu to 100%. The culprit seems to be the lock and synchronization processing within AbstractBrokerFactory.newBroker(..). According to sections 5.9.1 and 5.9.2 in the JPA specification, it looks like OpenJPA is attempting to do too much management of the created EntityManagers. Within a Container-managed environment, the Container takes care of the lifecycle of the EntityManagers. So, there does not seem to be a need to do the findBroker(..) invocation, nor is there a need to keep track of the created EntityManagers (_brokers) so that they can be closed when the Factory is closed.
> Once we have verified these changes, there may be others that are needed. But, we have to get by this bottleneck first before going to the next layer...
> Kevin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.