You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-java@ibatis.apache.org by Brian Parkinson <pa...@avaning.com> on 2008/01/09 23:16:34 UTC

Performance question

Hello:

 

I am using iBATIS, and it's working out well for me, but have run into a
performance issue, and wondering if anyone can help.

 

The object I am trying to persist is pretty complicated - it's an object
which contains a bunch of child objects, which contain children, etc.
The save method in the DAO (I am using the Spring DAO base class) is
pretty long and complicated, as it has to save each child object (or
list of child objects) separately. Example code is at the bottom of this
email.

 

I have put the whole thing into a transaction as well as doing the whole
thing in a batch, which I expect should have some performance
improvements.

 

When I run a performance test, I am finding that the insert (doSave) is
extremely slow - averaging about 100 ms (this after "priming" the test
with 1000 transactions, and running the test over an additional 1000).

 

So, my question is - can anyone make any suggestions on how to speed
this up - clearly 100ms is too long a time to persist this data.

 

Any help would be greatly appreciated.


Regards,

 

parki...

 

 

      private void doSave(Thermostat thermostat) throws Exception

      {

            try

            {           

                  getSqlMapClient().startTransaction();

                  getSqlMapClient().startBatch();

                  

                  int thermostatId = (Integer)
getSqlMapClientTemplate().insert("Thermostat.insert", thermostat);

                  

                  Location location = thermostat.getLocation();

                  location.setThermostatId(thermostatId);

                  getSqlMapClientTemplate().insert("Location.insert",
location);

 

                  Setting setting = thermostat.getSetting();

                  setting.setThermostatId(thermostatId);

                  getSqlMapClientTemplate().insert("Setting.insert",
setting);

                  

                  ContractorInfo contractorInfo =
thermostat.getContractorInfo();

                  contractorInfo.setThermostatId(thermostatId);

 
getSqlMapClientTemplate().insert("ContractorInfo.insert",
contractorInfo);

 

                  List<EquipmentInfo> equipmentInfos =
thermostat.getEquipmentInfos();

                  for(EquipmentInfo equipmentInfo : equipmentInfos)

                  {

                        equipmentInfo.setThermostatId(thermostatId);

 
getSqlMapClientTemplate().insert("EquipmentInfo.insert", equipmentInfo);

                  }

                  

                  InternetConfig internetConfig =
thermostat.getInternetConfig();

                  internetConfig.setThermostatId(thermostatId);

 
getSqlMapClientTemplate().insert("InternetConfig.insert",
internetConfig);

 

                  WifiConfig wifiConfig = thermostat.getWifiConfig();

                  wifiConfig.setThermostatId(thermostatId);

                  getSqlMapClientTemplate().insert("WifiConfig.insert",
wifiConfig);

                  

                  List<Zone> zones = thermostat.getZones();

                  for(Zone zone : zones)

                  {

                        zone.setThermostatId(thermostatId);

                        getSqlMapClientTemplate().insert("Zone.insert",
zone);

                  }

                  

                  Program program = thermostat.getProgram();

                  program.setThermostatId(thermostatId);

                  

                  int programId = (Integer)
getSqlMapClientTemplate().insert("Program.insert", program);

                  

                  List<CalendarEvent> calendarEvents =
program.getCalendarEvents();

                  for(CalendarEvent calendarEvent : calendarEvents)

                  {

                        calendarEvent.setProgramId(programId);

 
getSqlMapClientTemplate().insert("CalendarEvent.insert", calendarEvent);

                  }

                  

                  List<Climate> climates = program.getClimates();

                  for(Climate climate : climates)

                  {

                        climate.setProgramId(programId);

                        int climateId = (Integer)
getSqlMapClientTemplate().insert("Climate.insert", climate);

                        

                        List<ClimateZone> climateZones =
climate.getClimateZones();

                        for(ClimateZone climateZone : climateZones)

                        {

                              climateZone.setClimateId(climateId);

                              climateZone.setProgramId(programId);

 
getSqlMapClientTemplate().insert("ClimateZone.insert", climateZone);

                        }

                  }

                  

                  List<Period> periods = program.getPeriods();

                  for(Period period : periods)

                  {

                        period.setProgramId(programId);

 
getSqlMapClientTemplate().insert("Period.insert", period);

                  }

                  

                  getSqlMapClient().executeBatch();

                  getSqlMapClient().commitTransaction();

            }

            finally

            {

                  getSqlMapClient().endTransaction();

            }

      }

      

 

 

 


Re: Performance question

Posted by Christopher Lamey <cl...@localmatters.com>.
On 1/10/08 2:52 PM, "Brian Parkinson" <pa...@avaning.com> wrote:

> I wasn't quite sure in your answer whether the startBatch/executeBatch
> should work properly under the setup I have now - I was assuming that
> the performance problems I encountered were related (in part at least)
> to the fact that the statements weren't being batched, but run
> sequentially and so making many calls to the db. I'm just a little
> confused on this point.

Yes, it sounds to me like you haven't been batching up until now, so you
should see some performance improvement with the new setup.  Keep in mind
that too many inserts in a batch can actually be slower than no batch at
all.

Cheers,
Chris


RE: Performance question

Posted by Brian Parkinson <pa...@avaning.com>.
Thanks Christopher!

I'll rip out the startTransaction, commit and endTransaction stuff from
the code (now that Spring is taking care of this).

One final question if you don't mind:

I wasn't quite sure in your answer whether the startBatch/executeBatch
should work properly under the setup I have now - I was assuming that
the performance problems I encountered were related (in part at least)
to the fact that the statements weren't being batched, but run
sequentially and so making many calls to the db. I'm just a little
confused on this point.

I'm off to run some performance tests.

Thanks again - very cool progress - spring and iBATIS are great.

parki...


> 3. Given that there is transaction support, then I'm assuming that
both:
> 
> getSqlMapClient().startTransaction();
> and
> getSqlMapClient().startBatch();
> 
> will be properly honoured.

Actually, they won't.  The Spring SqlMapClientTemplate essentially does
a no-op on these calls - it assumes an external transaction manager will
handle everything for you outside of the code.  So what you can do is
just remove them from your code (including the commit and finally {
endTransaction }).

With the Spring declarative transaction handling, you can assume that
those are being called for you.  And if you chain methods that meet the
pointcut criteria, they will be wrapped in the same transaction as the
calling method
- pretty neat stuff.

Re: Performance question

Posted by Christopher Lamey <cl...@localmatters.com>.
On 1/10/08 1:58 PM, "Brian Parkinson" <pa...@avaning.com> wrote:

> Hi Christopher -
> 
> Finally got a chance to look at adding transaction support to my iBATIS
> code using Spring 2.5.1.
> Needed to use spring.jar as well as aspectweaver.jar.

Yea, it needs that jar for the AOP stuff.
 
Your config looks ok.

> 1. Is this it? Seriously - how incredibly painless!!!

There's a reason Spring is so widely used, eh?

> 2. Can you see anything that's missing? I'm using the apache commons
> dbcp BasicDataSource, and so with this setup, I'm
> looking good to have connection pooling as well as transaction support.

I use DBCP just like that, so it should work.  If you're paranoid, you can
check the number of connections against your db - it should match the min
connections specified for DBCP.

Another thing you can do is enable JDBC logging with iBATIS.  Set java.sql
category to DEBUG level in your log4j config and you can track connections
being used and released.  Don't leave it at DEBUG for any performance runs
though, it writes out a ton of messages.

> 3. Given that there is transaction support, then I'm assuming that both:
> 
> getSqlMapClient().startTransaction();
> and
> getSqlMapClient().startBatch();
> 
> will be properly honoured.

Actually, they won't.  The Spring SqlMapClientTemplate essentially does a
no-op on these calls - it assumes an external transaction manager will
handle everything for you outside of the code.  So what you can do is just
remove them from your code (including the commit and finally {
endTransaction }).

With the Spring declarative transaction handling, you can assume that those
are being called for you.  And if you chain methods that meet the pointcut
criteria, they will be wrapped in the same transaction as the calling method
- pretty neat stuff.

When starting out, I usually stick this in the middle of my method after a
couple inserts that I know will write data:

if (true) {
    throw new RuntimeException("Force exception to test rollback");
}

That way I verify the inserts before the exception rollback correctly.

Cheers,
Chris


RE: Performance question

Posted by Brian Parkinson <pa...@avaning.com>.
Hi Christopher -

Finally got a chance to look at adding transaction support to my iBATIS
code using Spring 2.5.1.
Needed to use spring.jar as well as aspectweaver.jar.

I followed the docs you indicated below, and added the following:

1. Add in the transaction aware data source manager:

	<bean id="txManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager"
>
		<property name="dataSource" ref="dataSource"/>
	</bean>

2. Add in the advice to ignore get* methods, and wrap everything else:
	
	<tx:advice id="txAdvice" transaction-manager="txManager">
		<tx:attributes>
			<tx:method name="get*" read-only="true" />
			<tx:method name="*" />
		</tx:attributes>
	</tx:advice>

3. Weave the aspect into any DAO implementation object:
	
	<aop:config>
		<aop:pointcut id="daoServiceOperation"
expression="execution(* com.ecobee.foundation.dao.ibatis.*.*(..))" />
		<aop:advisor advice-ref="txAdvice"
pointcut-ref="daoServiceOperation" />
	</aop:config>

This works - when IO throw an exception in the middle of my DAO
implementation, there is a rollback
now.

A couple questions:

1. Is this it? Seriously - how incredibly painless!!!

2. Can you see anything that's missing? I'm using the apache commons
dbcp BasicDataSource, and so with this setup, I'm
looking good to have connection pooling as well as transaction support.

3. Given that there is transaction support, then I'm assuming that both:

getSqlMapClient().startTransaction();
and
getSqlMapClient().startBatch();

will be properly honoured.

Thanks for your help - this is all pretty magical. :)

Cheers,

parki....




 

-----Original Message-----
From: Christopher Lamey [mailto:clamey@localmatters.com] 
Sent: Wednesday, January 09, 2008 5:52 PM
To: user-java@ibatis.apache.org
Subject: Re: Performance question

Those kind of days are no fun, that's for sure:)

Take a look at the Spring docs for an example of the transaction
management config.  Section 9.5.2 has an example:

http://static.springframework.org/spring/docs/2.5.x/reference/transactio
n.ht
ml#transaction-declarative-first-example

Basically, you setup a tx:advice that setups up a transaction manager to
work on specified methods.  Then you have an aop:config that specifies
an expression pattern to apply against a set of classes which ties in
the tx:advice.  The DataSourceTransactionManager transaction manager
essentially wraps around the DataSource for you.  There are other
transaction managers if you want to use them.

Their docs do a detailed job describing what's going on, it's very easy
and nice because your DAO code doesn't have to worry about explicitly
handling transactions and batches.

BTW, this style of transaction management declaration was new for Spring
2.0.  So you'll see a different style for any older examples.  This new
way is better, so ignore the old ones.

Cheers,
Chris

On 1/9/08 3:43 PM, "Brian Parkinson" <pa...@avaning.com> wrote:

> Thanks Christoper - it's end of day here (frustrating one :-) and so 
> I'll tackle these tomorrow - appreciated.
> 
> Below are my spring config setup - if anyone can point me directions 
> on how to set up the transaction handling in Spring (which I do *not* 
> have) this would also be appreciated.
> 
> Or any other suggestions - I'm new to iBATIS; it's been working out 
> great so far, so hopefully can get these numbers down to reasonable.
> 
> Cheers,
> 
> parki...
> 
> 
> <bean id="mapConfig"
> class="org.springframework.core.io.ClassPathResource">
> <constructor-arg>
> 
> <value>com/ecobee/foundation/dao/ibatis/SqlMapConfig.xml</value>
> </constructor-arg>
> </bean>
> 
> <bean id="dataSource"
> class="org.apache.commons.dbcp.BasicDataSource" 
> destroy-method="close"> <property name="driverClassName"
> value="com.mysql.jdbc.Driver" />
> <property name="url" value="jdbc:mysql:///ecobee" /> <property 
> name="username" value="ecobee" /> <property name="password" 
> value="ecobee" /> <property name="initialSize" value="2" /> <property 
> name="maxActive" value="5" /> <property name="maxIdle" value="2" /> 
> </bean>
> 
> <bean id="sqlMapClient"
> class="org.springframework.orm.ibatis.SqlMapClientFactoryBean">
> <property name="dataSource" ref="dataSource" /> <property 
> name="configLocation" ref="mapConfig" /> </bean>
> 
> -----Original Message-----
> From: Christopher Lamey [mailto:clamey@localmatters.com]
> Sent: Wednesday, January 09, 2008 4:36 PM
> To: user-java@ibatis.apache.org
> Subject: Re: Performance question
> 
> Hello,
> 
> A transaction can get to a certain size and start to build up a big 
> enough rollback segment or two that it slows the whole thing down.  So

> you might want to consider breaking up your inserts into smaller 
> transactions and see if that helps your speed.  I usually make the 
> size of the commit batches a runtime variable that I can play with to 
> see what's best.
> 
> Also, what kind of indexes are involved and how much data is in the 
> tables?
> It looks like you have a dozen or so tables with foreign keys, so 
> that's a bunch of indexes to update.  If you have more indexes on the 
> tables, it just adds to slow writes.
> 
> It also depends on how your db is configured.  For example, in 
> Postgres you can control whether or no fsyncs are forced, full page 
> writes occur, how big the write ahead buffers are, how many checkpoint

> logfile segments are used, etc.  It also depends on the I/O subsystem 
> where the actual db files live - the slower the disk, the slower your 
> db writes will be.
> 
> I'm also wondering why you have explicit transaction handling in your 
> DAO when you're using Spring?  Am I understanding your setup 
> correctly?
> IIRC,
> if you use the Spring SqlMapClient, iBATIS assumes you're using an 
> external transaction manager and so explicit calls to 
> startTransaction(), startBatch(), etc. have no effect.  And so if you 
> have no declarative transaction handling setup in Spring, there is no 
> transaction management going on.
> 
> Try throwing an new RuntimeException("Forcing transaction rollback") 
> halfway through that method.  See if everything before the exception 
> is throw is actually handled and rolled back.
> 
> Cheers,
> Chris
> 
> On 1/9/08 3:16 PM, "Brian Parkinson" <pa...@avaning.com> wrote:
> 
>> Hello:
>> 
>>  
>> 
>> I am using iBATIS, and it's working out well for me, but have run 
>> into
> a
>> performance issue, and wondering if anyone can help.
>> 
>>  
>> 
>> The object I am trying to persist is pretty complicated - it's an
> object
>> which contains a bunch of child objects, which contain children, etc.
>> The save method in the DAO (I am using the Spring DAO base class) is 
>> pretty long and complicated, as it has to save each child object (or 
>> list of child objects) separately. Example code is at the bottom of
> this
>> email.
>> 
>>  
>> 
>> I have put the whole thing into a transaction as well as doing the
> whole
>> thing in a batch, which I expect should have some performance 
>> improvements.
>> 
>>  
>> 
>> When I run a performance test, I am finding that the insert (doSave)
> is
>> extremely slow - averaging about 100 ms (this after "priming" the 
>> test with 1000 transactions, and running the test over an additional
1000).
>> 
>>  
>> 
>> So, my question is - can anyone make any suggestions on how to speed 
>> this up - clearly 100ms is too long a time to persist this data.
>> 
>>  
>> 
>> Any help would be greatly appreciated.
>> 
>> 
>> Regards,
>> 
>>  
>> 
>> parki...
>> 
>>  
>> 
>>  
>> 
>>       private void doSave(Thermostat thermostat) throws Exception
>> 
>>       {
>> 
>>             try
>> 
>>             {   
>> 
>>                   getSqlMapClient().startTransaction();
>> 
>>                   getSqlMapClient().startBatch();
>> 
>>                 
>> 
>>                   int thermostatId = (Integer) 
>> getSqlMapClientTemplate().insert("Thermostat.insert", thermostat);
>> 
>>                 
>> 
>>                   Location location = thermostat.getLocation();
>> 
>>                   location.setThermostatId(thermostatId);
>> 
>>                   getSqlMapClientTemplate().insert("Location.insert",
>> location);
>> 
>>  
>> 
>>                   Setting setting = thermostat.getSetting();
>> 
>>                   setting.setThermostatId(thermostatId);
>> 
>>                   getSqlMapClientTemplate().insert("Setting.insert",
>> setting);
>> 
>>                 
>> 
>>                   ContractorInfo contractorInfo = 
>> thermostat.getContractorInfo();
>> 
>>                   contractorInfo.setThermostatId(thermostatId);
>> 
>>  
>> getSqlMapClientTemplate().insert("ContractorInfo.insert",
>> contractorInfo);
>> 
>>  
>> 
>>                   List<EquipmentInfo> equipmentInfos = 
>> thermostat.getEquipmentInfos();
>> 
>>                   for(EquipmentInfo equipmentInfo : equipmentInfos)
>> 
>>                   {
>> 
>>                         equipmentInfo.setThermostatId(thermostatId);
>> 
>>  
>> getSqlMapClientTemplate().insert("EquipmentInfo.insert",
> equipmentInfo);
>> 
>>                   }
>> 
>>                 
>> 
>>                   InternetConfig internetConfig = 
>> thermostat.getInternetConfig();
>> 
>>                   internetConfig.setThermostatId(thermostatId);
>> 
>>  
>> getSqlMapClientTemplate().insert("InternetConfig.insert",
>> internetConfig);
>> 
>>  
>> 
>>                   WifiConfig wifiConfig = thermostat.getWifiConfig();
>> 
>>                   wifiConfig.setThermostatId(thermostatId);
>> 
>> 
> getSqlMapClientTemplate().insert("WifiConfig.insert",
>> wifiConfig);
>> 
>>                 
>> 
>>                   List<Zone> zones = thermostat.getZones();
>> 
>>                   for(Zone zone : zones)
>> 
>>                   {
>> 
>>                         zone.setThermostatId(thermostatId);
>> 
>> 
> getSqlMapClientTemplate().insert("Zone.insert",
>> zone);
>> 
>>                   }
>> 
>>                 
>> 
>>                   Program program = thermostat.getProgram();
>> 
>>                   program.setThermostatId(thermostatId);
>> 
>>                 
>> 
>>                   int programId = (Integer) 
>> getSqlMapClientTemplate().insert("Program.insert", program);
>> 
>>                 
>> 
>>                   List<CalendarEvent> calendarEvents = 
>> program.getCalendarEvents();
>> 
>>                   for(CalendarEvent calendarEvent : calendarEvents)
>> 
>>                   {
>> 
>>                         calendarEvent.setProgramId(programId);
>> 
>>  
>> getSqlMapClientTemplate().insert("CalendarEvent.insert",
> calendarEvent);
>> 
>>                   }
>> 
>>                 
>> 
>>                   List<Climate> climates = program.getClimates();
>> 
>>                   for(Climate climate : climates)
>> 
>>                   {
>> 
>>                         climate.setProgramId(programId);
>> 
>>                         int climateId = (Integer) 
>> getSqlMapClientTemplate().insert("Climate.insert", climate);
>> 
>>                 
>> 
>>                         List<ClimateZone> climateZones = 
>> climate.getClimateZones();
>> 
>>                         for(ClimateZone climateZone : climateZones)
>> 
>>                         {
>> 
>>                               climateZone.setClimateId(climateId);
>> 
>>                               climateZone.setProgramId(programId);
>> 
>>  
>> getSqlMapClientTemplate().insert("ClimateZone.insert", climateZone);
>> 
>>                         }
>> 
>>                   }
>> 
>>                 
>> 
>>                   List<Period> periods = program.getPeriods();
>> 
>>                   for(Period period : periods)
>> 
>>                   {
>> 
>>                         period.setProgramId(programId);
>> 
>>  
>> getSqlMapClientTemplate().insert("Period.insert", period);
>> 
>>                   }
>> 
>>                 
>> 
>>                   getSqlMapClient().executeBatch();
>> 
>>                   getSqlMapClient().commitTransaction();
>> 
>>             }
>> 
>>             finally
>> 
>>             {
>> 
>>                   getSqlMapClient().endTransaction();
>> 
>>             }
>> 
>>       }
>> 
>>       
>> 
>>  
>> 
>>  
>> 
>>  
>> 
> 


RE: Performance question

Posted by Brian Parkinson <pa...@avaning.com>.
Thanks Chris - I'll tackle that first thing.

I'm new to both Spring (2.0.7) and iBATIS (2.3.0.677) and very very
impressed with both - I have had no coding isses - really great stuff,
and everything "just works".

I'll keep you posted on how I make out.

Again, appreciated.

Regards,

parki...


-----Original Message-----
From: Christopher Lamey [mailto:clamey@localmatters.com] 
Sent: Wednesday, January 09, 2008 5:52 PM
To: user-java@ibatis.apache.org
Subject: Re: Performance question

Those kind of days are no fun, that's for sure:)

Take a look at the Spring docs for an example of the transaction
management
config.  Section 9.5.2 has an example:

http://static.springframework.org/spring/docs/2.5.x/reference/transactio
n.ht
ml#transaction-declarative-first-example

Basically, you setup a tx:advice that setups up a transaction manager to
work on specified methods.  Then you have an aop:config that specifies
an
expression pattern to apply against a set of classes which ties in the
tx:advice.  The DataSourceTransactionManager transaction manager
essentially
wraps around the DataSource for you.  There are other transaction
managers
if you want to use them.

Their docs do a detailed job describing what's going on, it's very easy
and
nice because your DAO code doesn't have to worry about explicitly
handling
transactions and batches.

BTW, this style of transaction management declaration was new for Spring
2.0.  So you'll see a different style for any older examples.  This new
way
is better, so ignore the old ones.

Cheers,
Chris

On 1/9/08 3:43 PM, "Brian Parkinson" <pa...@avaning.com> wrote:

> Thanks Christoper - it's end of day here (frustrating one :-) and so
> I'll tackle these tomorrow - appreciated.
> 
> Below are my spring config setup - if anyone can point me directions
on
> how to set up the transaction handling in Spring (which I do *not*
have)
> this would also be appreciated.
> 
> Or any other suggestions - I'm new to iBATIS; it's been working out
> great so far, so hopefully can get these numbers down to reasonable.
> 
> Cheers,
> 
> parki...
> 
> 
> <bean id="mapConfig"
> class="org.springframework.core.io.ClassPathResource">
> <constructor-arg>
> 
> <value>com/ecobee/foundation/dao/ibatis/SqlMapConfig.xml</value>
> </constructor-arg>
> </bean>
> 
> <bean id="dataSource"
> class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
> <property name="driverClassName"
> value="com.mysql.jdbc.Driver" />
> <property name="url" value="jdbc:mysql:///ecobee" />
> <property name="username" value="ecobee" />
> <property name="password" value="ecobee" /> 
> <property name="initialSize" value="2" />
> <property name="maxActive" value="5" /> 
> <property name="maxIdle" value="2" />
> </bean>
> 
> <bean id="sqlMapClient"
> class="org.springframework.orm.ibatis.SqlMapClientFactoryBean">
> <property name="dataSource" ref="dataSource" />
> <property name="configLocation" ref="mapConfig" />
> </bean>
> 
> -----Original Message-----
> From: Christopher Lamey [mailto:clamey@localmatters.com]
> Sent: Wednesday, January 09, 2008 4:36 PM
> To: user-java@ibatis.apache.org
> Subject: Re: Performance question
> 
> Hello,
> 
> A transaction can get to a certain size and start to build up a big
> enough
> rollback segment or two that it slows the whole thing down.  So you
> might
> want to consider breaking up your inserts into smaller transactions
and
> see
> if that helps your speed.  I usually make the size of the commit
batches
> a
> runtime variable that I can play with to see what's best.
> 
> Also, what kind of indexes are involved and how much data is in the
> tables?
> It looks like you have a dozen or so tables with foreign keys, so
that's
> a
> bunch of indexes to update.  If you have more indexes on the tables,
it
> just
> adds to slow writes.
> 
> It also depends on how your db is configured.  For example, in
Postgres
> you
> can control whether or no fsyncs are forced, full page writes occur,
how
> big
> the write ahead buffers are, how many checkpoint logfile segments are
> used,
> etc.  It also depends on the I/O subsystem where the actual db files
> live -
> the slower the disk, the slower your db writes will be.
> 
> I'm also wondering why you have explicit transaction handling in your
> DAO
> when you're using Spring?  Am I understanding your setup correctly?
> IIRC,
> if you use the Spring SqlMapClient, iBATIS assumes you're using an
> external
> transaction manager and so explicit calls to startTransaction(),
> startBatch(), etc. have no effect.  And so if you have no declarative
> transaction handling setup in Spring, there is no transaction
management
> going on.
> 
> Try throwing an new RuntimeException("Forcing transaction rollback")
> halfway
> through that method.  See if everything before the exception is throw
is
> actually handled and rolled back.
> 
> Cheers,
> Chris
> 
> On 1/9/08 3:16 PM, "Brian Parkinson" <pa...@avaning.com> wrote:
> 
>> Hello:
>> 
>>  
>> 
>> I am using iBATIS, and it's working out well for me, but have run
into
> a
>> performance issue, and wondering if anyone can help.
>> 
>>  
>> 
>> The object I am trying to persist is pretty complicated - it's an
> object
>> which contains a bunch of child objects, which contain children, etc.
>> The save method in the DAO (I am using the Spring DAO base class) is
>> pretty long and complicated, as it has to save each child object (or
>> list of child objects) separately. Example code is at the bottom of
> this
>> email.
>> 
>>  
>> 
>> I have put the whole thing into a transaction as well as doing the
> whole
>> thing in a batch, which I expect should have some performance
>> improvements.
>> 
>>  
>> 
>> When I run a performance test, I am finding that the insert (doSave)
> is
>> extremely slow - averaging about 100 ms (this after "priming" the
test
>> with 1000 transactions, and running the test over an additional
1000).
>> 
>>  
>> 
>> So, my question is - can anyone make any suggestions on how to speed
>> this up - clearly 100ms is too long a time to persist this data.
>> 
>>  
>> 
>> Any help would be greatly appreciated.
>> 
>> 
>> Regards,
>> 
>>  
>> 
>> parki...
>> 
>>  
>> 
>>  
>> 
>>       private void doSave(Thermostat thermostat) throws Exception
>> 
>>       {
>> 
>>             try
>> 
>>             {   
>> 
>>                   getSqlMapClient().startTransaction();
>> 
>>                   getSqlMapClient().startBatch();
>> 
>>                 
>> 
>>                   int thermostatId = (Integer)
>> getSqlMapClientTemplate().insert("Thermostat.insert", thermostat);
>> 
>>                 
>> 
>>                   Location location = thermostat.getLocation();
>> 
>>                   location.setThermostatId(thermostatId);
>> 
>>                   getSqlMapClientTemplate().insert("Location.insert",
>> location);
>> 
>>  
>> 
>>                   Setting setting = thermostat.getSetting();
>> 
>>                   setting.setThermostatId(thermostatId);
>> 
>>                   getSqlMapClientTemplate().insert("Setting.insert",
>> setting);
>> 
>>                 
>> 
>>                   ContractorInfo contractorInfo =
>> thermostat.getContractorInfo();
>> 
>>                   contractorInfo.setThermostatId(thermostatId);
>> 
>>  
>> getSqlMapClientTemplate().insert("ContractorInfo.insert",
>> contractorInfo);
>> 
>>  
>> 
>>                   List<EquipmentInfo> equipmentInfos =
>> thermostat.getEquipmentInfos();
>> 
>>                   for(EquipmentInfo equipmentInfo : equipmentInfos)
>> 
>>                   {
>> 
>>                         equipmentInfo.setThermostatId(thermostatId);
>> 
>>  
>> getSqlMapClientTemplate().insert("EquipmentInfo.insert",
> equipmentInfo);
>> 
>>                   }
>> 
>>                 
>> 
>>                   InternetConfig internetConfig =
>> thermostat.getInternetConfig();
>> 
>>                   internetConfig.setThermostatId(thermostatId);
>> 
>>  
>> getSqlMapClientTemplate().insert("InternetConfig.insert",
>> internetConfig);
>> 
>>  
>> 
>>                   WifiConfig wifiConfig = thermostat.getWifiConfig();
>> 
>>                   wifiConfig.setThermostatId(thermostatId);
>> 
>> 
> getSqlMapClientTemplate().insert("WifiConfig.insert",
>> wifiConfig);
>> 
>>                 
>> 
>>                   List<Zone> zones = thermostat.getZones();
>> 
>>                   for(Zone zone : zones)
>> 
>>                   {
>> 
>>                         zone.setThermostatId(thermostatId);
>> 
>> 
> getSqlMapClientTemplate().insert("Zone.insert",
>> zone);
>> 
>>                   }
>> 
>>                 
>> 
>>                   Program program = thermostat.getProgram();
>> 
>>                   program.setThermostatId(thermostatId);
>> 
>>                 
>> 
>>                   int programId = (Integer)
>> getSqlMapClientTemplate().insert("Program.insert", program);
>> 
>>                 
>> 
>>                   List<CalendarEvent> calendarEvents =
>> program.getCalendarEvents();
>> 
>>                   for(CalendarEvent calendarEvent : calendarEvents)
>> 
>>                   {
>> 
>>                         calendarEvent.setProgramId(programId);
>> 
>>  
>> getSqlMapClientTemplate().insert("CalendarEvent.insert",
> calendarEvent);
>> 
>>                   }
>> 
>>                 
>> 
>>                   List<Climate> climates = program.getClimates();
>> 
>>                   for(Climate climate : climates)
>> 
>>                   {
>> 
>>                         climate.setProgramId(programId);
>> 
>>                         int climateId = (Integer)
>> getSqlMapClientTemplate().insert("Climate.insert", climate);
>> 
>>                 
>> 
>>                         List<ClimateZone> climateZones =
>> climate.getClimateZones();
>> 
>>                         for(ClimateZone climateZone : climateZones)
>> 
>>                         {
>> 
>>                               climateZone.setClimateId(climateId);
>> 
>>                               climateZone.setProgramId(programId);
>> 
>>  
>> getSqlMapClientTemplate().insert("ClimateZone.insert", climateZone);
>> 
>>                         }
>> 
>>                   }
>> 
>>                 
>> 
>>                   List<Period> periods = program.getPeriods();
>> 
>>                   for(Period period : periods)
>> 
>>                   {
>> 
>>                         period.setProgramId(programId);
>> 
>>  
>> getSqlMapClientTemplate().insert("Period.insert", period);
>> 
>>                   }
>> 
>>                 
>> 
>>                   getSqlMapClient().executeBatch();
>> 
>>                   getSqlMapClient().commitTransaction();
>> 
>>             }
>> 
>>             finally
>> 
>>             {
>> 
>>                   getSqlMapClient().endTransaction();
>> 
>>             }
>> 
>>       }
>> 
>>       
>> 
>>  
>> 
>>  
>> 
>>  
>> 
> 


Re: Performance question

Posted by Christopher Lamey <cl...@localmatters.com>.
Those kind of days are no fun, that's for sure:)

Take a look at the Spring docs for an example of the transaction management
config.  Section 9.5.2 has an example:

http://static.springframework.org/spring/docs/2.5.x/reference/transaction.ht
ml#transaction-declarative-first-example

Basically, you setup a tx:advice that setups up a transaction manager to
work on specified methods.  Then you have an aop:config that specifies an
expression pattern to apply against a set of classes which ties in the
tx:advice.  The DataSourceTransactionManager transaction manager essentially
wraps around the DataSource for you.  There are other transaction managers
if you want to use them.

Their docs do a detailed job describing what's going on, it's very easy and
nice because your DAO code doesn't have to worry about explicitly handling
transactions and batches.

BTW, this style of transaction management declaration was new for Spring
2.0.  So you'll see a different style for any older examples.  This new way
is better, so ignore the old ones.

Cheers,
Chris

On 1/9/08 3:43 PM, "Brian Parkinson" <pa...@avaning.com> wrote:

> Thanks Christoper - it's end of day here (frustrating one :-) and so
> I'll tackle these tomorrow - appreciated.
> 
> Below are my spring config setup - if anyone can point me directions on
> how to set up the transaction handling in Spring (which I do *not* have)
> this would also be appreciated.
> 
> Or any other suggestions - I'm new to iBATIS; it's been working out
> great so far, so hopefully can get these numbers down to reasonable.
> 
> Cheers,
> 
> parki...
> 
> 
> <bean id="mapConfig"
> class="org.springframework.core.io.ClassPathResource">
> <constructor-arg>
> 
> <value>com/ecobee/foundation/dao/ibatis/SqlMapConfig.xml</value>
> </constructor-arg>
> </bean>
> 
> <bean id="dataSource"
> class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
> <property name="driverClassName"
> value="com.mysql.jdbc.Driver" />
> <property name="url" value="jdbc:mysql:///ecobee" />
> <property name="username" value="ecobee" />
> <property name="password" value="ecobee" /> 
> <property name="initialSize" value="2" />
> <property name="maxActive" value="5" /> 
> <property name="maxIdle" value="2" />
> </bean>
> 
> <bean id="sqlMapClient"
> class="org.springframework.orm.ibatis.SqlMapClientFactoryBean">
> <property name="dataSource" ref="dataSource" />
> <property name="configLocation" ref="mapConfig" />
> </bean>
> 
> -----Original Message-----
> From: Christopher Lamey [mailto:clamey@localmatters.com]
> Sent: Wednesday, January 09, 2008 4:36 PM
> To: user-java@ibatis.apache.org
> Subject: Re: Performance question
> 
> Hello,
> 
> A transaction can get to a certain size and start to build up a big
> enough
> rollback segment or two that it slows the whole thing down.  So you
> might
> want to consider breaking up your inserts into smaller transactions and
> see
> if that helps your speed.  I usually make the size of the commit batches
> a
> runtime variable that I can play with to see what's best.
> 
> Also, what kind of indexes are involved and how much data is in the
> tables?
> It looks like you have a dozen or so tables with foreign keys, so that's
> a
> bunch of indexes to update.  If you have more indexes on the tables, it
> just
> adds to slow writes.
> 
> It also depends on how your db is configured.  For example, in Postgres
> you
> can control whether or no fsyncs are forced, full page writes occur, how
> big
> the write ahead buffers are, how many checkpoint logfile segments are
> used,
> etc.  It also depends on the I/O subsystem where the actual db files
> live -
> the slower the disk, the slower your db writes will be.
> 
> I'm also wondering why you have explicit transaction handling in your
> DAO
> when you're using Spring?  Am I understanding your setup correctly?
> IIRC,
> if you use the Spring SqlMapClient, iBATIS assumes you're using an
> external
> transaction manager and so explicit calls to startTransaction(),
> startBatch(), etc. have no effect.  And so if you have no declarative
> transaction handling setup in Spring, there is no transaction management
> going on.
> 
> Try throwing an new RuntimeException("Forcing transaction rollback")
> halfway
> through that method.  See if everything before the exception is throw is
> actually handled and rolled back.
> 
> Cheers,
> Chris
> 
> On 1/9/08 3:16 PM, "Brian Parkinson" <pa...@avaning.com> wrote:
> 
>> Hello:
>> 
>>  
>> 
>> I am using iBATIS, and it's working out well for me, but have run into
> a
>> performance issue, and wondering if anyone can help.
>> 
>>  
>> 
>> The object I am trying to persist is pretty complicated - it's an
> object
>> which contains a bunch of child objects, which contain children, etc.
>> The save method in the DAO (I am using the Spring DAO base class) is
>> pretty long and complicated, as it has to save each child object (or
>> list of child objects) separately. Example code is at the bottom of
> this
>> email.
>> 
>>  
>> 
>> I have put the whole thing into a transaction as well as doing the
> whole
>> thing in a batch, which I expect should have some performance
>> improvements.
>> 
>>  
>> 
>> When I run a performance test, I am finding that the insert (doSave)
> is
>> extremely slow - averaging about 100 ms (this after "priming" the test
>> with 1000 transactions, and running the test over an additional 1000).
>> 
>>  
>> 
>> So, my question is - can anyone make any suggestions on how to speed
>> this up - clearly 100ms is too long a time to persist this data.
>> 
>>  
>> 
>> Any help would be greatly appreciated.
>> 
>> 
>> Regards,
>> 
>>  
>> 
>> parki...
>> 
>>  
>> 
>>  
>> 
>>       private void doSave(Thermostat thermostat) throws Exception
>> 
>>       {
>> 
>>             try
>> 
>>             {   
>> 
>>                   getSqlMapClient().startTransaction();
>> 
>>                   getSqlMapClient().startBatch();
>> 
>>                 
>> 
>>                   int thermostatId = (Integer)
>> getSqlMapClientTemplate().insert("Thermostat.insert", thermostat);
>> 
>>                 
>> 
>>                   Location location = thermostat.getLocation();
>> 
>>                   location.setThermostatId(thermostatId);
>> 
>>                   getSqlMapClientTemplate().insert("Location.insert",
>> location);
>> 
>>  
>> 
>>                   Setting setting = thermostat.getSetting();
>> 
>>                   setting.setThermostatId(thermostatId);
>> 
>>                   getSqlMapClientTemplate().insert("Setting.insert",
>> setting);
>> 
>>                 
>> 
>>                   ContractorInfo contractorInfo =
>> thermostat.getContractorInfo();
>> 
>>                   contractorInfo.setThermostatId(thermostatId);
>> 
>>  
>> getSqlMapClientTemplate().insert("ContractorInfo.insert",
>> contractorInfo);
>> 
>>  
>> 
>>                   List<EquipmentInfo> equipmentInfos =
>> thermostat.getEquipmentInfos();
>> 
>>                   for(EquipmentInfo equipmentInfo : equipmentInfos)
>> 
>>                   {
>> 
>>                         equipmentInfo.setThermostatId(thermostatId);
>> 
>>  
>> getSqlMapClientTemplate().insert("EquipmentInfo.insert",
> equipmentInfo);
>> 
>>                   }
>> 
>>                 
>> 
>>                   InternetConfig internetConfig =
>> thermostat.getInternetConfig();
>> 
>>                   internetConfig.setThermostatId(thermostatId);
>> 
>>  
>> getSqlMapClientTemplate().insert("InternetConfig.insert",
>> internetConfig);
>> 
>>  
>> 
>>                   WifiConfig wifiConfig = thermostat.getWifiConfig();
>> 
>>                   wifiConfig.setThermostatId(thermostatId);
>> 
>> 
> getSqlMapClientTemplate().insert("WifiConfig.insert",
>> wifiConfig);
>> 
>>                 
>> 
>>                   List<Zone> zones = thermostat.getZones();
>> 
>>                   for(Zone zone : zones)
>> 
>>                   {
>> 
>>                         zone.setThermostatId(thermostatId);
>> 
>> 
> getSqlMapClientTemplate().insert("Zone.insert",
>> zone);
>> 
>>                   }
>> 
>>                 
>> 
>>                   Program program = thermostat.getProgram();
>> 
>>                   program.setThermostatId(thermostatId);
>> 
>>                 
>> 
>>                   int programId = (Integer)
>> getSqlMapClientTemplate().insert("Program.insert", program);
>> 
>>                 
>> 
>>                   List<CalendarEvent> calendarEvents =
>> program.getCalendarEvents();
>> 
>>                   for(CalendarEvent calendarEvent : calendarEvents)
>> 
>>                   {
>> 
>>                         calendarEvent.setProgramId(programId);
>> 
>>  
>> getSqlMapClientTemplate().insert("CalendarEvent.insert",
> calendarEvent);
>> 
>>                   }
>> 
>>                 
>> 
>>                   List<Climate> climates = program.getClimates();
>> 
>>                   for(Climate climate : climates)
>> 
>>                   {
>> 
>>                         climate.setProgramId(programId);
>> 
>>                         int climateId = (Integer)
>> getSqlMapClientTemplate().insert("Climate.insert", climate);
>> 
>>                 
>> 
>>                         List<ClimateZone> climateZones =
>> climate.getClimateZones();
>> 
>>                         for(ClimateZone climateZone : climateZones)
>> 
>>                         {
>> 
>>                               climateZone.setClimateId(climateId);
>> 
>>                               climateZone.setProgramId(programId);
>> 
>>  
>> getSqlMapClientTemplate().insert("ClimateZone.insert", climateZone);
>> 
>>                         }
>> 
>>                   }
>> 
>>                 
>> 
>>                   List<Period> periods = program.getPeriods();
>> 
>>                   for(Period period : periods)
>> 
>>                   {
>> 
>>                         period.setProgramId(programId);
>> 
>>  
>> getSqlMapClientTemplate().insert("Period.insert", period);
>> 
>>                   }
>> 
>>                 
>> 
>>                   getSqlMapClient().executeBatch();
>> 
>>                   getSqlMapClient().commitTransaction();
>> 
>>             }
>> 
>>             finally
>> 
>>             {
>> 
>>                   getSqlMapClient().endTransaction();
>> 
>>             }
>> 
>>       }
>> 
>>       
>> 
>>  
>> 
>>  
>> 
>>  
>> 
> 


RE: Performance question

Posted by Brian Parkinson <pa...@avaning.com>.
Thanks Christoper - it's end of day here (frustrating one :-) and so
I'll tackle these tomorrow - appreciated.

Below are my spring config setup - if anyone can point me directions on
how to set up the transaction handling in Spring (which I do *not* have)
this would also be appreciated.

Or any other suggestions - I'm new to iBATIS; it's been working out
great so far, so hopefully can get these numbers down to reasonable.

Cheers,

parki...


	<bean id="mapConfig"
class="org.springframework.core.io.ClassPathResource">
		<constructor-arg>
	
<value>com/ecobee/foundation/dao/ibatis/SqlMapConfig.xml</value>
		</constructor-arg>
	</bean>
	 
	<bean id="dataSource"
class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
		<property name="driverClassName"
value="com.mysql.jdbc.Driver" />
		<property name="url" value="jdbc:mysql:///ecobee" />
		<property name="username" value="ecobee" />
		<property name="password" value="ecobee" />	
		<property name="initialSize" value="2" />
		<property name="maxActive" value="5" />	
		<property name="maxIdle" value="2" />
	</bean>

	<bean id="sqlMapClient"
class="org.springframework.orm.ibatis.SqlMapClientFactoryBean">
		<property name="dataSource" ref="dataSource" />
		<property name="configLocation" ref="mapConfig" />
	</bean>

-----Original Message-----
From: Christopher Lamey [mailto:clamey@localmatters.com] 
Sent: Wednesday, January 09, 2008 4:36 PM
To: user-java@ibatis.apache.org
Subject: Re: Performance question

Hello,

A transaction can get to a certain size and start to build up a big
enough
rollback segment or two that it slows the whole thing down.  So you
might
want to consider breaking up your inserts into smaller transactions and
see
if that helps your speed.  I usually make the size of the commit batches
a
runtime variable that I can play with to see what's best.

Also, what kind of indexes are involved and how much data is in the
tables?
It looks like you have a dozen or so tables with foreign keys, so that's
a
bunch of indexes to update.  If you have more indexes on the tables, it
just
adds to slow writes.

It also depends on how your db is configured.  For example, in Postgres
you
can control whether or no fsyncs are forced, full page writes occur, how
big
the write ahead buffers are, how many checkpoint logfile segments are
used,
etc.  It also depends on the I/O subsystem where the actual db files
live -
the slower the disk, the slower your db writes will be.

I'm also wondering why you have explicit transaction handling in your
DAO
when you're using Spring?  Am I understanding your setup correctly?
IIRC,
if you use the Spring SqlMapClient, iBATIS assumes you're using an
external
transaction manager and so explicit calls to startTransaction(),
startBatch(), etc. have no effect.  And so if you have no declarative
transaction handling setup in Spring, there is no transaction management
going on.

Try throwing an new RuntimeException("Forcing transaction rollback")
halfway
through that method.  See if everything before the exception is throw is
actually handled and rolled back.

Cheers,
Chris

On 1/9/08 3:16 PM, "Brian Parkinson" <pa...@avaning.com> wrote:

> Hello:
> 
>  
> 
> I am using iBATIS, and it's working out well for me, but have run into
a
> performance issue, and wondering if anyone can help.
> 
>  
> 
> The object I am trying to persist is pretty complicated - it's an
object
> which contains a bunch of child objects, which contain children, etc.
> The save method in the DAO (I am using the Spring DAO base class) is
> pretty long and complicated, as it has to save each child object (or
> list of child objects) separately. Example code is at the bottom of
this
> email.
> 
>  
> 
> I have put the whole thing into a transaction as well as doing the
whole
> thing in a batch, which I expect should have some performance
> improvements.
> 
>  
> 
> When I run a performance test, I am finding that the insert (doSave)
is
> extremely slow - averaging about 100 ms (this after "priming" the test
> with 1000 transactions, and running the test over an additional 1000).
> 
>  
> 
> So, my question is - can anyone make any suggestions on how to speed
> this up - clearly 100ms is too long a time to persist this data.
> 
>  
> 
> Any help would be greatly appreciated.
> 
> 
> Regards,
> 
>  
> 
> parki...
> 
>  
> 
>  
> 
>       private void doSave(Thermostat thermostat) throws Exception
> 
>       {
> 
>             try
> 
>             {    
> 
>                   getSqlMapClient().startTransaction();
> 
>                   getSqlMapClient().startBatch();
> 
>                  
> 
>                   int thermostatId = (Integer)
> getSqlMapClientTemplate().insert("Thermostat.insert", thermostat);
> 
>                  
> 
>                   Location location = thermostat.getLocation();
> 
>                   location.setThermostatId(thermostatId);
> 
>                   getSqlMapClientTemplate().insert("Location.insert",
> location);
> 
>  
> 
>                   Setting setting = thermostat.getSetting();
> 
>                   setting.setThermostatId(thermostatId);
> 
>                   getSqlMapClientTemplate().insert("Setting.insert",
> setting);
> 
>                  
> 
>                   ContractorInfo contractorInfo =
> thermostat.getContractorInfo();
> 
>                   contractorInfo.setThermostatId(thermostatId);
> 
>  
> getSqlMapClientTemplate().insert("ContractorInfo.insert",
> contractorInfo);
> 
>  
> 
>                   List<EquipmentInfo> equipmentInfos =
> thermostat.getEquipmentInfos();
> 
>                   for(EquipmentInfo equipmentInfo : equipmentInfos)
> 
>                   {
> 
>                         equipmentInfo.setThermostatId(thermostatId);
> 
>  
> getSqlMapClientTemplate().insert("EquipmentInfo.insert",
equipmentInfo);
> 
>                   }
> 
>                  
> 
>                   InternetConfig internetConfig =
> thermostat.getInternetConfig();
> 
>                   internetConfig.setThermostatId(thermostatId);
> 
>  
> getSqlMapClientTemplate().insert("InternetConfig.insert",
> internetConfig);
> 
>  
> 
>                   WifiConfig wifiConfig = thermostat.getWifiConfig();
> 
>                   wifiConfig.setThermostatId(thermostatId);
> 
>
getSqlMapClientTemplate().insert("WifiConfig.insert",
> wifiConfig);
> 
>                  
> 
>                   List<Zone> zones = thermostat.getZones();
> 
>                   for(Zone zone : zones)
> 
>                   {
> 
>                         zone.setThermostatId(thermostatId);
> 
>
getSqlMapClientTemplate().insert("Zone.insert",
> zone);
> 
>                   }
> 
>                  
> 
>                   Program program = thermostat.getProgram();
> 
>                   program.setThermostatId(thermostatId);
> 
>                  
> 
>                   int programId = (Integer)
> getSqlMapClientTemplate().insert("Program.insert", program);
> 
>                  
> 
>                   List<CalendarEvent> calendarEvents =
> program.getCalendarEvents();
> 
>                   for(CalendarEvent calendarEvent : calendarEvents)
> 
>                   {
> 
>                         calendarEvent.setProgramId(programId);
> 
>  
> getSqlMapClientTemplate().insert("CalendarEvent.insert",
calendarEvent);
> 
>                   }
> 
>                  
> 
>                   List<Climate> climates = program.getClimates();
> 
>                   for(Climate climate : climates)
> 
>                   {
> 
>                         climate.setProgramId(programId);
> 
>                         int climateId = (Integer)
> getSqlMapClientTemplate().insert("Climate.insert", climate);
> 
>                  
> 
>                         List<ClimateZone> climateZones =
> climate.getClimateZones();
> 
>                         for(ClimateZone climateZone : climateZones)
> 
>                         {
> 
>                               climateZone.setClimateId(climateId);
> 
>                               climateZone.setProgramId(programId);
> 
>  
> getSqlMapClientTemplate().insert("ClimateZone.insert", climateZone);
> 
>                         }
> 
>                   }
> 
>                  
> 
>                   List<Period> periods = program.getPeriods();
> 
>                   for(Period period : periods)
> 
>                   {
> 
>                         period.setProgramId(programId);
> 
>  
> getSqlMapClientTemplate().insert("Period.insert", period);
> 
>                   }
> 
>                  
> 
>                   getSqlMapClient().executeBatch();
> 
>                   getSqlMapClient().commitTransaction();
> 
>             }
> 
>             finally
> 
>             {
> 
>                   getSqlMapClient().endTransaction();
> 
>             }
> 
>       }
> 
>       
> 
>  
> 
>  
> 
>  
> 


Re: Performance question

Posted by Christopher Lamey <cl...@localmatters.com>.
Hello,

A transaction can get to a certain size and start to build up a big enough
rollback segment or two that it slows the whole thing down.  So you might
want to consider breaking up your inserts into smaller transactions and see
if that helps your speed.  I usually make the size of the commit batches a
runtime variable that I can play with to see what's best.

Also, what kind of indexes are involved and how much data is in the tables?
It looks like you have a dozen or so tables with foreign keys, so that's a
bunch of indexes to update.  If you have more indexes on the tables, it just
adds to slow writes.

It also depends on how your db is configured.  For example, in Postgres you
can control whether or no fsyncs are forced, full page writes occur, how big
the write ahead buffers are, how many checkpoint logfile segments are used,
etc.  It also depends on the I/O subsystem where the actual db files live -
the slower the disk, the slower your db writes will be.

I'm also wondering why you have explicit transaction handling in your DAO
when you're using Spring?  Am I understanding your setup correctly?  IIRC,
if you use the Spring SqlMapClient, iBATIS assumes you're using an external
transaction manager and so explicit calls to startTransaction(),
startBatch(), etc. have no effect.  And so if you have no declarative
transaction handling setup in Spring, there is no transaction management
going on.

Try throwing an new RuntimeException("Forcing transaction rollback") halfway
through that method.  See if everything before the exception is throw is
actually handled and rolled back.

Cheers,
Chris

On 1/9/08 3:16 PM, "Brian Parkinson" <pa...@avaning.com> wrote:

> Hello:
> 
>  
> 
> I am using iBATIS, and it's working out well for me, but have run into a
> performance issue, and wondering if anyone can help.
> 
>  
> 
> The object I am trying to persist is pretty complicated - it's an object
> which contains a bunch of child objects, which contain children, etc.
> The save method in the DAO (I am using the Spring DAO base class) is
> pretty long and complicated, as it has to save each child object (or
> list of child objects) separately. Example code is at the bottom of this
> email.
> 
>  
> 
> I have put the whole thing into a transaction as well as doing the whole
> thing in a batch, which I expect should have some performance
> improvements.
> 
>  
> 
> When I run a performance test, I am finding that the insert (doSave) is
> extremely slow - averaging about 100 ms (this after "priming" the test
> with 1000 transactions, and running the test over an additional 1000).
> 
>  
> 
> So, my question is - can anyone make any suggestions on how to speed
> this up - clearly 100ms is too long a time to persist this data.
> 
>  
> 
> Any help would be greatly appreciated.
> 
> 
> Regards,
> 
>  
> 
> parki...
> 
>  
> 
>  
> 
>       private void doSave(Thermostat thermostat) throws Exception
> 
>       {
> 
>             try
> 
>             {    
> 
>                   getSqlMapClient().startTransaction();
> 
>                   getSqlMapClient().startBatch();
> 
>                  
> 
>                   int thermostatId = (Integer)
> getSqlMapClientTemplate().insert("Thermostat.insert", thermostat);
> 
>                  
> 
>                   Location location = thermostat.getLocation();
> 
>                   location.setThermostatId(thermostatId);
> 
>                   getSqlMapClientTemplate().insert("Location.insert",
> location);
> 
>  
> 
>                   Setting setting = thermostat.getSetting();
> 
>                   setting.setThermostatId(thermostatId);
> 
>                   getSqlMapClientTemplate().insert("Setting.insert",
> setting);
> 
>                  
> 
>                   ContractorInfo contractorInfo =
> thermostat.getContractorInfo();
> 
>                   contractorInfo.setThermostatId(thermostatId);
> 
>  
> getSqlMapClientTemplate().insert("ContractorInfo.insert",
> contractorInfo);
> 
>  
> 
>                   List<EquipmentInfo> equipmentInfos =
> thermostat.getEquipmentInfos();
> 
>                   for(EquipmentInfo equipmentInfo : equipmentInfos)
> 
>                   {
> 
>                         equipmentInfo.setThermostatId(thermostatId);
> 
>  
> getSqlMapClientTemplate().insert("EquipmentInfo.insert", equipmentInfo);
> 
>                   }
> 
>                  
> 
>                   InternetConfig internetConfig =
> thermostat.getInternetConfig();
> 
>                   internetConfig.setThermostatId(thermostatId);
> 
>  
> getSqlMapClientTemplate().insert("InternetConfig.insert",
> internetConfig);
> 
>  
> 
>                   WifiConfig wifiConfig = thermostat.getWifiConfig();
> 
>                   wifiConfig.setThermostatId(thermostatId);
> 
>                   getSqlMapClientTemplate().insert("WifiConfig.insert",
> wifiConfig);
> 
>                  
> 
>                   List<Zone> zones = thermostat.getZones();
> 
>                   for(Zone zone : zones)
> 
>                   {
> 
>                         zone.setThermostatId(thermostatId);
> 
>                         getSqlMapClientTemplate().insert("Zone.insert",
> zone);
> 
>                   }
> 
>                  
> 
>                   Program program = thermostat.getProgram();
> 
>                   program.setThermostatId(thermostatId);
> 
>                  
> 
>                   int programId = (Integer)
> getSqlMapClientTemplate().insert("Program.insert", program);
> 
>                  
> 
>                   List<CalendarEvent> calendarEvents =
> program.getCalendarEvents();
> 
>                   for(CalendarEvent calendarEvent : calendarEvents)
> 
>                   {
> 
>                         calendarEvent.setProgramId(programId);
> 
>  
> getSqlMapClientTemplate().insert("CalendarEvent.insert", calendarEvent);
> 
>                   }
> 
>                  
> 
>                   List<Climate> climates = program.getClimates();
> 
>                   for(Climate climate : climates)
> 
>                   {
> 
>                         climate.setProgramId(programId);
> 
>                         int climateId = (Integer)
> getSqlMapClientTemplate().insert("Climate.insert", climate);
> 
>                  
> 
>                         List<ClimateZone> climateZones =
> climate.getClimateZones();
> 
>                         for(ClimateZone climateZone : climateZones)
> 
>                         {
> 
>                               climateZone.setClimateId(climateId);
> 
>                               climateZone.setProgramId(programId);
> 
>  
> getSqlMapClientTemplate().insert("ClimateZone.insert", climateZone);
> 
>                         }
> 
>                   }
> 
>                  
> 
>                   List<Period> periods = program.getPeriods();
> 
>                   for(Period period : periods)
> 
>                   {
> 
>                         period.setProgramId(programId);
> 
>  
> getSqlMapClientTemplate().insert("Period.insert", period);
> 
>                   }
> 
>                  
> 
>                   getSqlMapClient().executeBatch();
> 
>                   getSqlMapClient().commitTransaction();
> 
>             }
> 
>             finally
> 
>             {
> 
>                   getSqlMapClient().endTransaction();
> 
>             }
> 
>       }
> 
>       
> 
>  
> 
>  
> 
>  
>