You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-java@ibatis.apache.org by Kyohyoh Choh <CH...@jp.ibm.com> on 2007/04/21 02:10:42 UTC

How to set datasource thread save?




Hello all

Our application use Spring+iBATIS, need to switch among multiple
datasources based on user's request datasourceid parameter. We are testing
setDataSource() method of SqlMapClientDAOSupport class. We found if we set
our DAO class which extends SqlMapClientDAOSupport to be singleton
class(Spring's default value), the datasource can't set correctly when
users access concurrently. If we set singleton=false, everything goes will.
The user's datasourceid parameter is hold as thread local. Does this mean
SqlMapClientDAOSupport is not thread save? This is my first time to use
Spring+iBATIS, if I miss something, please forgive me and point out.

Sippete of our DAO class

            DataSource datasource = null;

            //get datasource from ApplicationContext
            datasource = (DataSource) ctx.getBean(datasourceID);

            //set datasource
            SqlMapClientTemplate sqlMapClientTem = null;
            sqlMapClientTem = getSqlMapClientTemplate();
            sqlMapClientTem.setDataSource(datasource);
            domain = sqlMapClientTem.queryForObject(selectID , inobj);

Thank you


RoutableSqlMapClient Example

Posted by Chris Lamey <cl...@localmatters.com>.
Hello,

Here is how I've got dynamic DataSources working in my app.  It is based
on the idea of Spring's AbstractRoutingDataSource class, but operates
with SqlMapClients instead to avoid caching issues in iBATIS.  Since
SqlMapClient doesn't segment its cache based on DataSources, a single
SqlMapClient will merge all the results from multiple DataSources into
one cache.

Attached are two classes, RoutableSqlMapClient and VendorContextHolder.
RoutableSqlMapClient is a class that implements the ExtendedSqlMapClient
interface but delegates all calls to other SqlMapClients.  It does the
delegation by using VendorContextHolder, which holds a static
ThreadLocal variable, which for my needs is represents a VendorTypes
enum.  This ThreadLocal variable is a key into a Map of target delegate
SqlMapClient objects.  Each consumer calling into this code will pass
along a vendor, which is used to set the ThreadLocal variable in
VendorContextHolder.

Here is how the relevant Spring config looks:

<!--
    Pulls the VendorOne DataSource from JNDI and sets up the
    SqlMapClient for VendorOne, the other vendors look the same.
    
    I also use Spring's 2.x AOP declarative transaction management
    around each vendor's DataSource, but I'm leaving that out for this
    example.

    Also, in my case, all my SqlMapClients use the same sqlmap config.
-->
<bean id="vendorOneDataSource"   
        class="org.springframework.jndi.JndiObjectFactoryBean">
    <property name="jndiName">
        <value>java:/comp/env/jdbc/VendorOne</value>
    </property>
</bean>
<bean id="vendorOneSqlMapClient"   
        class="org.springframework.orm.ibatis.SqlMapClientFactoryBean">
    <property name="configLocation"
        value="classpath:nontology/sqlmap-config.xml"/>
    <property name="dataSource" ref="vendorOneDataSource"/>
</bean>

<!--
    Sets up the RoutingSqlMapClient with a Map of vendor specific
    SqlMapClient beans and it keyed by the VendorTypes enum.
-->
<bean id="sqlMapClient"
        class="com.localmatters.bo.core.util.RoutingSqlMapClient">
    <property name="targetSqlMapClients">
    <map key-type="com.localmatters.bo.core.commons.VendorTypes">
        <entry key="VendorOne" value-ref="vendorOneSqlMapClient"/>
        <entry key="VendorTwo" value-ref="vendorTwoSqlMapClient"/>
        <entry key="VendorThree" value-ref="vendorThreeSqlMapClient"/>
    </map>
    </property>
</bean>

<!--
    Creates an Abator-generated DAO bean and sets the RoutableSqlMap.
-->
<bean id="userDAO"  
        class="com.localmatters.bo.core.nontology.dao.UserDAOImpl">
    <property name="sqlMapClient" ref="sqlMapClient"/>
</bean>

The consumers of this API use a Service Locator/Manager that wraps
Spring.  When a thread wants a DAO, the code looks like this:

public UserDAO getUserDAO(VendorTypes vendor) {
    VendorContextHolder.setVendorType(vendor);
    return (UserDAO) context.getBean("userDAO");
}

It is entirely possible to not have a Service Locater/Manager and have
the consumer set the VendorContextHolder directly.  The Service
Locator/Manager exists in this code because of legacy reasons.

At this point, the DAO just calls the ExtendedSqlMapClient methods as
normal, but they get routed by RoutableSqlMapClient to a vendor specific
SqlMapClient, complete with its own DataSource, TransactionManager,
cache, etc.

There is a limitation with this approach that should be evaluated before
use.  As you can see, this scheme sets a ThreadLocal variable that
determines the DataSource to use for the remainder of that Thread's
execution.  So in my case, if the Thread needs a different vendor, it
must make that call to the Service Locator with the different vendor.
But this means a Thread can effectively only hit one DataSource at a
time, it is *not possible* to intermix access to multiple DataSources
from the same Thread.

For example, this won't work:

    UserDAO vendorOneDAO =
        ServiceLocator.getUserDAO(VendorTypes.VendorOne);
    UserDAO vendorTwoDAO = 
        ServiceLocator.getUserDAO(VendorTYpes.VendorTwo);

    vendorOneDAO.update(vendorOneUser);
    vendorTwoDAO.update(vendorTwoUser);

Because the DataSource is determined on a per-Thread basis, the second
call to getUserDAO will set the vendor to VendorTwo.  Plus there's
really only one UserDAO in the Spring Context.  Regardless, the first
call to update(vendorOneUser) will actually hit the VendorTwo
DataSource.  And really, if you're using Spring correctly, you probably
aren't using a Service Locator/Manager, so it may not matter.  In that
case, you probably have a single DAO pointer and would use it something
like this:

    VendorContextHolder.set(VendorTypes.VendorOne);
    vendorDAO.update(vendorOneUser);
    VendorContextHolder.set(VendorTypes.VendorTwo);
    vendorDAO.update(vendorTwoUser);

Which kind of sucks because your consumer code has to know about the
VendorContextHolder, but it works.

My consumer applications have no need to support the intermixing of
multiple DataSources by a single Thread, so I didn't consider it until
after I was done.  Offhand, I'd say this RoutableSqlMapClient approach
(and by extension, Spring's AbstractRoutingDataSource) cannot support
intermixing.

I've also attached a servlet Filter that looks for a 'vendor' parameter
in the request URL and sets the VendorContextHolder accordingly.  I use
this for a fat client that talks to a remoting service via a webapp, but
it should work for all-HTML webapps too.

Hopefully all that made some sense.

Cheers,
Chris

RE: How to set datasource thread save?

Posted by Justin Musgrove <JM...@fcci-group.com>.
Chris,

Thanks for your explanation.  Would you mind posting code/xml config
files of how you used AbstractRoutableDataSource?

Thanks a bunch...

Justin

-----Original Message-----
From: Chris Lamey [mailto:clamey@localmatters.com] 
Sent: Saturday, April 21, 2007 4:43 PM
To: user-java@ibatis.apache.org
Subject: RE: How to set datasource thread save?

The Spring SqlMapClient is defintely thread safe.  I've had thousands of
concurrent threads hitting a single SqlMapClient and each one has proper
transaction handling and caching.  I use Abator generated DAOs, not the
Spring ones, but my guess is that the Spring DAOs are thread-safe too
because they are widely used.

The issue is that using iBATIS with Spring doesn't have a clear 'best
practice' way of dynamically switching between DataSources at runtime.

My guess is that if you go with a singleton DAO instance, have hundreds
of concurrent threads hitting it, and change the datasource at runtime
for each thread, you are then changing out the datasource for all
threads hitting the DAO at that point.

What you are doing now is creating a new DAO instance for each call per
thread.  This allows each thread to have its own DAO on its stack, which
limits the touch points betwen threads.  I remember that you said you
are not using caching in the SqlMapClient, which is good, because if you
were this setup would not work.  The SqlMapClient doesn't know you are
switching out DataSources and would merge all the results into one cache
for all DataSources.

Creating a new DAO for each call per thread is not something I would
personally want in my code.  In my opinion, there's no reason DAOs can't
be singetons and to have so many created and thrown away is an
indication of a design problem.  Plus I have lots of caching in iBATIS
and cannot afford to stop using the cache.  So I created that
RoutableSqlMapClient class based on Spring's AbstractRoutableDataSource,
which dynamically routes calls by the singleton DAOs to any number of
SqlMapClients, each with their own DataSource, cache, and transaction
handling.  Any number of threads with the same DataSource can be in
their respective SqlMapClient/DataSource combo, as long as the
DataSource doesn't have to change within the run of a thread, all is
well.

Cheers,
Chris

-----Original Message-----
From: Kyohyoh Choh [mailto:CHOH@jp.ibm.com]
Sent: Fri 4/20/2007 6:10 PM
To: user-java@ibatis.apache.org
Subject: How to set datasource thread save?
 




Hello all

Our application use Spring+iBATIS, need to switch among multiple
datasources based on user's request datasourceid parameter. We are
testing
setDataSource() method of SqlMapClientDAOSupport class. We found if we
set our DAO class which extends SqlMapClientDAOSupport to be singleton
class(Spring's default value), the datasource can't set correctly when
users access concurrently. If we set singleton=false, everything goes
will.
The user's datasourceid parameter is hold as thread local. Does this
mean SqlMapClientDAOSupport is not thread save? This is my first time to
use
Spring+iBATIS, if I miss something, please forgive me and point out.

Sippete of our DAO class

            DataSource datasource = null;

            //get datasource from ApplicationContext
            datasource = (DataSource) ctx.getBean(datasourceID);

            //set datasource
            SqlMapClientTemplate sqlMapClientTem = null;
            sqlMapClientTem = getSqlMapClientTemplate();
            sqlMapClientTem.setDataSource(datasource);
            domain = sqlMapClientTem.queryForObject(selectID , inobj);

Thank you



RE: How to set datasource thread save?

Posted by Kyohyoh Choh <CH...@jp.ibm.com>.



Hello
Yes, if we use setDatasource() of sqlMapClientTemplate for a singleton DAO,
datasource is changed for all threads of the DAO instance.
Routing many SqlMapClient to a single DAO is interesting, would you please
post your config file? Also I want to know whether you specified
sqlmapconfig file for each sqlmapclient or not.

Thank you very much



                                                                           
             "Chris Lamey"                                                 
             <clamey@localmatt                                             
             ers.com>                                                   To 
                                       <us...@ibatis.apache.org>       
             2007/04/22 05:42                                           cc 
                                                                           
                                                                   Subject 
             Please respond to         RE: How to set datasource thread    
             user-java@ibatis.         save?                               
                apache.org                                                 
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




The Spring SqlMapClient is defintely thread safe.  I've had thousands of
concurrent threads hitting a single SqlMapClient and each one has proper
transaction handling and caching.  I use Abator generated DAOs, not the
Spring ones, but my guess is that the Spring DAOs are thread-safe too
because they are widely used.

The issue is that using iBATIS with Spring doesn't have a clear 'best
practice' way of dynamically switching between DataSources at runtime.

My guess is that if you go with a singleton DAO instance, have hundreds of
concurrent threads hitting it, and change the datasource at runtime for
each thread, you are then changing out the datasource for all threads
hitting the DAO at that point.

What you are doing now is creating a new DAO instance for each call per
thread.  This allows each thread to have its own DAO on its stack, which
limits the touch points betwen threads.  I remember that you said you are
not using caching in the SqlMapClient, which is good, because if you were
this setup would not work.  The SqlMapClient doesn't know you are switching
out DataSources and would merge all the results into one cache for all
DataSources.

Creating a new DAO for each call per thread is not something I would
personally want in my code.  In my opinion, there's no reason DAOs can't be
singetons and to have so many created and thrown away is an indication of a
design problem.  Plus I have lots of caching in iBATIS and cannot afford to
stop using the cache.  So I created that RoutableSqlMapClient class based
on Spring's AbstractRoutableDataSource, which dynamically routes calls by
the singleton DAOs to any number of SqlMapClients, each with their own
DataSource, cache, and transaction handling.  Any number of threads with
the same DataSource can be in their respective SqlMapClient/DataSource
combo, as long as the DataSource doesn't have to change within the run of a
thread, all is well.

Cheers,
Chris







Hello all

Our application use Spring+iBATIS, need to switch among multiple
datasources based on user's request datasourceid parameter. We are testing
setDataSource() method of SqlMapClientDAOSupport class. We found if we set
our DAO class which extends SqlMapClientDAOSupport to be singleton
class(Spring's default value), the datasource can't set correctly when
users access concurrently. If we set singleton=false, everything goes will.
The user's datasourceid parameter is hold as thread local. Does this mean
SqlMapClientDAOSupport is not thread save? This is my first time to use
Spring+iBATIS, if I miss something, please forgive me and point out.

Sippete of our DAO class

            DataSource datasource = null;

            //get datasource from ApplicationContext
            datasource = (DataSource) ctx.getBean(datasourceID);

            //set datasource
            SqlMapClientTemplate sqlMapClientTem = null;
            sqlMapClientTem = getSqlMapClientTemplate();
            sqlMapClientTem.setDataSource(datasource);
            domain = sqlMapClientTem.queryForObject(selectID , inobj);

Thank you


[添付ファイル "winmail.dat" は Kyohyoh Choh/Japan/IBM が削除しました]


RE: How to set datasource thread save?

Posted by Chris Lamey <cl...@localmatters.com>.
The Spring SqlMapClient is defintely thread safe.  I've had thousands of concurrent threads hitting a single SqlMapClient and each one has proper transaction handling and caching.  I use Abator generated DAOs, not the Spring ones, but my guess is that the Spring DAOs are thread-safe too because they are widely used.

The issue is that using iBATIS with Spring doesn't have a clear 'best practice' way of dynamically switching between DataSources at runtime.

My guess is that if you go with a singleton DAO instance, have hundreds of concurrent threads hitting it, and change the datasource at runtime for each thread, you are then changing out the datasource for all threads hitting the DAO at that point.

What you are doing now is creating a new DAO instance for each call per thread.  This allows each thread to have its own DAO on its stack, which limits the touch points betwen threads.  I remember that you said you are not using caching in the SqlMapClient, which is good, because if you were this setup would not work.  The SqlMapClient doesn't know you are switching out DataSources and would merge all the results into one cache for all DataSources.

Creating a new DAO for each call per thread is not something I would personally want in my code.  In my opinion, there's no reason DAOs can't be singetons and to have so many created and thrown away is an indication of a design problem.  Plus I have lots of caching in iBATIS and cannot afford to stop using the cache.  So I created that RoutableSqlMapClient class based on Spring's AbstractRoutableDataSource, which dynamically routes calls by the singleton DAOs to any number of SqlMapClients, each with their own DataSource, cache, and transaction handling.  Any number of threads with the same DataSource can be in their respective SqlMapClient/DataSource combo, as long as the DataSource doesn't have to change within the run of a thread, all is well.

Cheers,
Chris

-----Original Message-----
From: Kyohyoh Choh [mailto:CHOH@jp.ibm.com]
Sent: Fri 4/20/2007 6:10 PM
To: user-java@ibatis.apache.org
Subject: How to set datasource thread save?
 




Hello all

Our application use Spring+iBATIS, need to switch among multiple
datasources based on user's request datasourceid parameter. We are testing
setDataSource() method of SqlMapClientDAOSupport class. We found if we set
our DAO class which extends SqlMapClientDAOSupport to be singleton
class(Spring's default value), the datasource can't set correctly when
users access concurrently. If we set singleton=false, everything goes will.
The user's datasourceid parameter is hold as thread local. Does this mean
SqlMapClientDAOSupport is not thread save? This is my first time to use
Spring+iBATIS, if I miss something, please forgive me and point out.

Sippete of our DAO class

            DataSource datasource = null;

            //get datasource from ApplicationContext
            datasource = (DataSource) ctx.getBean(datasourceID);

            //set datasource
            SqlMapClientTemplate sqlMapClientTem = null;
            sqlMapClientTem = getSqlMapClientTemplate();
            sqlMapClientTem.setDataSource(datasource);
            domain = sqlMapClientTem.queryForObject(selectID , inobj);

Thank you