Re: Performance question [Problem solved!]

[Wernke zur Borg <w....@rheagroup.com>]

I solved the problem by wrapping the entire processing into one 
transaction, following "User-Defined Transaction Scope" in 
http://cayenne.apache.org/doc/understanding-transactions.html.

         // embed the whole thing in one single transaction
         DataDomain d = getRuntime().getDataDomain();
         Transaction tx = d.createTransaction();
         Transaction.bindThreadTransaction(tx);

         try {
                 // do all processing here
         } finally {
             Transaction.bindThreadTransaction(null);
         }

No need to worry about rollbacks as everything is read-only.

 >>> Processing time has gone down from one hour to one minute !!!

Thanks to everybody,
Wernke


On 2012-01-25 10:01, Wernke zur Borg wrote:
>
> To give you a picture here is a screenshot of the profiler showing 
> that one connection is opened and closed for every query.
>
>
>
> The question I do not understand is why does DataNode.performQueries() 
> unconditionally close the current connection?
>
> Otherwise I could probably provide a custom DataSource class 
> overriding getConnection() to re-use connections.
>
> Wernke
>
>
> On 2012-01-24 17:11, Wernke zur Borg wrote:
>> Dear list,
>>
>> I have another question. My application converts a huge MS Access
>> database to a set of XML files by traversing some really big tables with
>> Select queries and also querying related tables to collect the
>> information needed for the output. Performance is horribly bad, so I did
>> some profiling and found that it spends 75% of its CPU time in
>> org.apache.cayenne.access.ExternalTransaction.close() and another 21% in
>> org.apache.cayenne.conn.DriverDataSource.getConnection(). Only 2% are
>> spent in SelectAction.performAction().
>>
>> So it appears that it opens and closes a new connection for every single
>> select query. I must be doing something wrong here! Where can I tell it
>> to re-use connections? Or make one big transaction as it is read-only
>> anyway? Any other hints? I hope I can reduce the processing time to 4%
>> of what it is now...
>>
>> Thanks,
>> Wernke
>>



This email (including any attachments) may contain confidential and/or privileged
information or information otherwise protected from disclosure.  If you are not the
intended recipient, please notify the sender immediately, do not copy this message
or any attachments and do not use it or any purpose or disclose its content to any
person, but delete this message and any attachments from your system.
RHEA System S.A. (RHEA) disclaims any and all liability if this email transmission
was virus corrupted, altered or falsified.

If a Non Disclosure Agreement (NDA) exists between RHEA and the receiving
organization, then all contents shall be considered proprietary and covered under
the NDA.