You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ojb-user@db.apache.org by "Plummer, Greg" <Gr...@LibertyMutual.com> on 2003/12/22 22:38:01 UTC
RE: Suspected Junk Mail: Re: [rc5 / bug / MetadataManager] Memory leak using dynamic mapping o n per thread bases
Hi Armin,
Thanks very much for taking such quick action on this!!! We'll be
testing out the fix today.
We may very well be doing something wrong in our usage. We have a
requirement that when our users are making changes to the data, they
need to be able to view the data that is currently in production. We
implemented this by having 2 databases, a "staging" database, where the
changes are made, and a "release" database, that stores what is
currently in production. The 2 are almost identical, with the exception
that the staging db uses identity columns and the release db does not
(because the ids need to remain in sync between the 2 DBs). So we are
using 2 profiles, but we only have 1 repository-user.xml file. We make a
copy of the original and dynamically turn off the access=readonly for
the release db. If there's a better way to accomplish this, I would
welcome any advice, although we don't time for any major changes right
now.
Greg
-----Original Message-----
From: Armin Waibel [mailto:armin@code-au-lait.de]
Sent: Friday, December 19, 2003 12:42 PM
To: OJB Users List; Plummer, Greg; Janssen, Roger
Subject: Suspected Junk Mail: Re: [rc5 / bug / MetadataManager] Memory
leak using dynamic mapping o n per thread bases
Hi Roger/Greg,
first, thanks for pointing to this bug, it should be fixed now. Get
latest from CVS (trunk or OJB_BRANCH_1_0).
http://nagoya.apache.org/eyebrowse/ReadMsg?listName=ojb-dev@db.apache.or
g&msgNo=5592
The test case I use is called
...broker.metadata.MetadataMultithreadedTest
see OJB test suite classes. Test pass without problems
(you can play with loops/thread number of the test).
What I don't understand is how you could find the leak, because the
intention of 'per-thread-changes' use is *not* to copy the repository
for each call/thread, rather to manage a bunch of different repositories
(e.g. by using MetadataManager#addProfile/loadProfile... methods).
Why you need a 'fresh'copy for each thread/call? Could be a performance
problem - maybe I misinterpret your needs.
regards,
Armin
Janssen, Roger wrote:
> hi,
>
> executing the code below should reproduce the memory leak [of course
> swap the class name for a class name present in youe repository
> mapping :)], so best is to execute it in a loop:
>
> MetadataManager mm = MetadataManager.getInstance();
> // tell the manager to use per thread mode
> mm.setEnablePerThreadChanges(true);
>
> // e.g we get a coppy of the global repository
> DescriptorRepository dr = mm.copyOfGlobalRepository();
>
> // now we can manipulate the metadata of the copy
> // .... do nothing
>
> // set the changed repository for this thread
> mm.setDescriptor(dr);
>
> org.apache.ojb.broker.query.Query q =
> (org.apache.ojb.broker.query.Query)
> new
> QueryByCriteria(Class.forName("ibanx.doc.DocumentIF"), null, true);
>
> PersistenceBroker brokerImpl =
> PersistenceBrokerFactory.defaultPersistenceBroker();
>
> Collection c = brokerImpl.getCollectionByQuery(q);
>
> // cleanup
> brokerImpl.close();
> brokerImpl = null;
> if(c!=null)
> {
> c.clear();
> c = null;
> }
> dr = null;
> mm = null;
>
>
> in the example above, i actually do not even change the mapping, but
> every time within the loop, a copy is made of the global repository,
> and set using ThreadLocal in MetadataManager, this action does not
> release the memory of the previous, but does add another copy of the
> repository.
>
> even if this code is executed not directly within a loop, but within a
> loop, a thread is started that executes this code snippet, after which
> the thread ends, then also the memory is not released.
>
> Why it is not released (does threadlocal not detect the
> stopping/destruction of the thread, or is threadlocal not able to
> identify the copy of the repository (i tried adding a hashcode method
> but that did not work) that has to be released) i do not know! I do
> know that the usage of ThreadLocal is tricky, and that this is not the
> first time memory leaks occur because of usage of this class.
>
> Hope this helps a bit.
>
> Roger Janssen
>
>
>
> **********************************************************************
> ***
> The information contained in this communication is confidential and is
> intended solely for the use of the individual or entity to whom it is
> addressed.You should not copy, disclose or distribute this
communication
> without the authority of iBanx bv. iBanx bv is neither liable for
> the proper and complete transmission of the information has been
maintained
> nor that the communication is free of viruses, interceptions or
interference.
>
> If you are not the intended recipient of this communication please
> return the communication to the sender and delete and destroy all
> copies.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-user-help@db.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org
Re: Suspected Junk Mail: Re: [rc5 / bug / MetadataManager] Memory
leak using dynamic mapping o n per thread bases
Posted by Armin Waibel <ar...@code-au-lait.de>.
Hi Greg,
Plummer, Greg wrote:
> Hi Armin,
>
> Thanks very much for taking such quick action on this!!! We'll be
> testing out the fix today.
Great! Let me known if there are still problems.
>
> We may very well be doing something wrong in our usage. We have a
> requirement that when our users are making changes to the data, they
> need to be able to view the data that is currently in production. We
> implemented this by having 2 databases, a "staging" database, where the
> changes are made, and a "release" database, that stores what is
> currently in production. The 2 are almost identical, with the exception
> that the staging db uses identity columns and the release db does not
> (because the ids need to remain in sync between the 2 DBs). So we are
> using 2 profiles, but we only have 1 repository-user.xml file. We make a
> copy of the original and dynamically turn off the access=readonly for
> the release db.
Do you make one copy of repository, or a copy on each call?
2 profiles and one repository file seems ok.
My intention is:
On start of OJB (or first time OJB was called) enable per thread
changes. Make a copy of DescriptorRepository (I assume
access="readwrite" is set by default) and use
MM.addProfile("release", copy);
to add the first profile.
Get another copy and dynamically turn on the access=readonly
for all classes (or contrary if 'readonly' is set in repository file).
MM.addProfile("staging", anotherCopy);
Now you can load the according profile for each thread/user, e.g. the
'release' profile by calling
MM.loadProfile("release");
Then all PB instances of the calling thread using the 'release' profile.
Don't know if this will fit your requirements.
regards,
Armin
> If there's a better way to accomplish this, I would
> welcome any advice, although we don't time for any major changes right
> now.
>
> Greg
>
> -----Original Message-----
> From: Armin Waibel [mailto:armin@code-au-lait.de]
> Sent: Friday, December 19, 2003 12:42 PM
> To: OJB Users List; Plummer, Greg; Janssen, Roger
> Subject: Suspected Junk Mail: Re: [rc5 / bug / MetadataManager] Memory
> leak using dynamic mapping o n per thread bases
>
>
> Hi Roger/Greg,
>
> first, thanks for pointing to this bug, it should be fixed now. Get
> latest from CVS (trunk or OJB_BRANCH_1_0).
>
> http://nagoya.apache.org/eyebrowse/ReadMsg?listName=ojb-dev@db.apache.or
> g&msgNo=5592
>
> The test case I use is called
> ...broker.metadata.MetadataMultithreadedTest
> see OJB test suite classes. Test pass without problems
> (you can play with loops/thread number of the test).
>
> What I don't understand is how you could find the leak, because the
> intention of 'per-thread-changes' use is *not* to copy the repository
> for each call/thread, rather to manage a bunch of different repositories
> (e.g. by using MetadataManager#addProfile/loadProfile... methods).
>
> Why you need a 'fresh'copy for each thread/call? Could be a performance
> problem - maybe I misinterpret your needs.
>
> regards,
> Armin
>
>
> Janssen, Roger wrote:
>
>
>>hi,
>>
>>executing the code below should reproduce the memory leak [of course
>>swap the class name for a class name present in youe repository
>>mapping :)], so best is to execute it in a loop:
>>
>> MetadataManager mm = MetadataManager.getInstance();
>> // tell the manager to use per thread mode
>> mm.setEnablePerThreadChanges(true);
>>
>> // e.g we get a coppy of the global repository
>> DescriptorRepository dr = mm.copyOfGlobalRepository();
>>
>> // now we can manipulate the metadata of the copy
>> // .... do nothing
>>
>> // set the changed repository for this thread
>> mm.setDescriptor(dr);
>>
>> org.apache.ojb.broker.query.Query q =
>> (org.apache.ojb.broker.query.Query)
>> new
>>QueryByCriteria(Class.forName("ibanx.doc.DocumentIF"), null, true);
>>
>> PersistenceBroker brokerImpl =
>>PersistenceBrokerFactory.defaultPersistenceBroker();
>>
>> Collection c = brokerImpl.getCollectionByQuery(q);
>>
>> // cleanup
>> brokerImpl.close();
>> brokerImpl = null;
>> if(c!=null)
>> {
>> c.clear();
>> c = null;
>> }
>> dr = null;
>> mm = null;
>>
>>
>>in the example above, i actually do not even change the mapping, but
>>every time within the loop, a copy is made of the global repository,
>>and set using ThreadLocal in MetadataManager, this action does not
>>release the memory of the previous, but does add another copy of the
>>repository.
>>
>>even if this code is executed not directly within a loop, but within a
>
>
>>loop, a thread is started that executes this code snippet, after which
>
>
>>the thread ends, then also the memory is not released.
>>
>>Why it is not released (does threadlocal not detect the
>>stopping/destruction of the thread, or is threadlocal not able to
>>identify the copy of the repository (i tried adding a hashcode method
>>but that did not work) that has to be released) i do not know! I do
>>know that the usage of ThreadLocal is tricky, and that this is not the
>
>
>>first time memory leaks occur because of usage of this class.
>>
>>Hope this helps a bit.
>>
>>Roger Janssen
>>
>>
>>
>>**********************************************************************
>>***
>>The information contained in this communication is confidential and is
>>intended solely for the use of the individual or entity to whom it is
>>addressed.You should not copy, disclose or distribute this
>
> communication
>
>>without the authority of iBanx bv. iBanx bv is neither liable for
>>the proper and complete transmission of the information has been
>
> maintained
>
>>nor that the communication is free of viruses, interceptions or
>
> interference.
>
>>If you are not the intended recipient of this communication please
>>return the communication to the sender and delete and destroy all
>>copies.
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
>>For additional commands, e-mail: ojb-user-help@db.apache.org
>>
>>
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-user-help@db.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org