Re: Using the Cayenne API to dynamically create DataDomains

[Michael Martineau <mi...@familypursuit.com>]

Andrus,

Thanks for the reply.  For those interested and for future reference, 
here is what I came up with.  It seems to work just fine.  The 
"template" datasouce is setup using the Modeler to use DBCP.  I'm new to 
Cayenne, so this may not be the best way. If anyone with more experience 
wants to comment, I would appreciate the feedback.

- Michael Martineau

public class Main4
{
     public static void main(String[] args) throws SQLException
     {
         Main4 main = new Main4();

         main.createNewDataDomain("db0");
         main.addUser("db0", "Michael");

         main.createNewDataDomain("db1");
         main.addUser("db1", "Rebekah");

         main.createNewDataDomain("db2");
         main.addUser("db2", "Andrew");

         main.createNewDataDomain("db3");
         main.addUser("db3", "Rachael");

         System.exit(0);
     }

     private void createNewDataDomain(String name) throws SQLException
     {
         String schema = "cayenne_" + name;
         DataDomain templateDomain = 
Configuration.getSharedConfiguration().getDomain("Cayenne_DBX");
         DataSource dataSource = 
templateDomain.getNode("Cayenne_DBXNode").getDataSource();

         DataNode dataNode = new DataNode("DynamicNode" + name);
         dataNode.setAdapter(new AutoAdapter(dataSource));
         dataNode.setDataSource(dataSource);

         MapLoader mapLoader = new MapLoader();
         DataMap dataMap = mapLoader.loadDataMap("Cayenne_DBXMap.map.xml");
         dataMap.setDefaultSchema(schema);

         Map dbEntities = dataMap.getDbEntityMap();
         for (Iterator entities = dbEntities.values().iterator(); 
entities.hasNext();)
         {
             DbEntity entity = (DbEntity) entities.next();
             entity.setSchema(schema);
         }

         dataNode.addDataMap(dataMap);

         DataDomain domainX = new DataDomain(name);
         domainX.addNode(dataNode);

         Configuration c = Configuration.getSharedConfiguration();
         c.addDomain(domainX);

     }

     private void addUser(String domain, String username)
     {
         ObjectContext context = DataContext.createDataContext(domain);

         User user = context.newObject(User.class);
         user.setName(username);
         context.commitChanges();

         List<User> users = (List<User>) context.performQuery(new 
SelectQuery(User.class));

         for (User u : users)
         {
             System.out.println(u.getName());
         }
     }
}





On 2/12/2010 4:40 AM, Andrus Adamchik wrote:
> Hi Michael,
>
> Cayenne was initially designed to run in a multi-stack scenario like 
> yours, so it is a good fit. Still since the scenario is not that 
> common, there is no prepackaged recipe. But you are on the right 
> track. See my comment below...
>
>> So, here are my questions:
>> 1. How do I use the Cayenne API to dynamically tie into the DBCP 
>> connection pool?
>
> http://cayenne.apache.org/doc30/dbcpdatasourcefactory.html
>
> The above gives you a DBCP connection pool in your "template" domain. 
> To share it between multiple domains, you need to reuse the DataSource 
> attached to the DataNode of the template domain. So in your example 
> you'd replace
>
>> DataSource dataSource = new PoolManager(...);
>
> with
>
> DataSource dataSource = templateDomain.getNode("XYZ").getDataSource();
>
> But that also means that you will need to clone the DataMap and reset 
> the schema for all DbEntities to "cayenne_" + userName, instead of 
> reusing them per your example below. (I actually suggest instead of 
> cloning, re-loading DataMap from XML using MapLoader, which will be 
> more reliable).
>
>> 2. Once I've done that, how do I dynamically switch schemas on a per 
>> HttpRequest basis?
>
> Many ways to do that. E.g. via a servlet filter.
>
> void doFilter(..) {
>    String userName = // get it from request somehow
>    Configuration c = Configuration.getSharedConfiguration();
>    DataDomain domain = c.getDomain(userName);
>    if(domain == null) {
>      // init on demand
>    }
>
>    BaseContext.bindThreadObjectContext(domain.createDataContext());
> }
>
> Later in your webapp code executed within request:
>
>    ObjectContext context = BaseContext .getThreadObjectContext();
>
>> 3. Will the solution work with multiple concurrent users each 
>> accessing their own database?
>
> Absolutely. The only code that may require some attention to thread 
> safety is "// init on demand" line above.
>
>> 4. Is Cayenne able to solve the memory consumption problem I'm having 
>> with Hibernate?  I.E., will Cayenne add little to no additional 
>> dedicated memory resources for every new DataDomain I create if I'm 
>> using a common database structure for each new database?
>
> Extra Cayenne-related memory consumption per user will be roughly 
> equal to the DataMap size. I suggest you put an upper cap on that, 
> e.g. by storing user stacks not in shared Configuration, but rather in 
> your own LRUMap, that will automatically kick out the stacks for 
> inactive users. Another thing you may do is to reduce the cache size 
> for each domain, by doing this:
>
> domain.getProperties().put("cayenne.DataRowStore.snapshot.size", "200");
>
> This will give you a fixed max memory profile.
>
> Hope this helps.
>
> Andrus
>
>
>
> On Feb 12, 2010, at 1:29 AM, Michael Martineau wrote:
>> Hello,
>>
>> I'm evaluating to see if Cayenne would be a good replacement for 
>> Hibernate in my current project.  I'm creating a website using tomcat 
>> where users register on the main website (e.g. www.mydomain.com).  
>> When they register, they select their own sub-domain name for a 
>> private database (e.g. myprivatedb.mydomain.com).  The web app then 
>> creates a new database instance for them using a common schema.  
>> Thus, each database has the exact same structure.  When each user 
>> goes to their own sub-domain and signs in, they are connecting to 
>> their private database (I use the sub-domain as a key to determine 
>> which database to use).  However, even though each user is accessing 
>> their own database, all users are sharing the same JVM tomcat 
>> instance.  Furthermore, all databases are hosted by the same Mysql 
>> server instance.
>>
>> After combing through the this email list I've been able to piece 
>> together a simple example of using the Modeler to create a base 
>> DataDomain/DataMap.  Then using the Cayenne API, I've managed to 
>> demonstrate to myself the ability to dynamically add new DataDomains 
>> that use the DataMap template created using the Modeler (see code 
>> below).  However, in my example, not only am I creating new 
>> DataDomains, I'm also creating a new connection pool for each 
>> DataDomain which is not really what I want.  I would like to use the 
>> commons-dbcp pool and have all users share the same pool of 
>> connections to the database server, even though they access different 
>> schemas (mysql databases).
>>
>> I've managed to make this work in Hibernate by implementing their 
>> ConnectionProvider interface.  Each new database configuration 
>> (SessionFactory) in Hibernate creates a ConnectionProvider which 
>> allowed me to change the "default" schema on the connection (that I 
>> get from the DBCP pool) just before handing it to Hibernate based on 
>> the website sub-domain.  The problem is that Hibernate must create a 
>> new SessionFactory for every database schema.  Unfortunately, this 
>> "SessionFactory" takes up a fixed amount of memory (5MB in my case) 
>> on the server for each new database regardless of whether the user 
>> who created the private database is signed in and using the website 
>> or not.  What I need is something (Cayenne?) that will take advantage 
>> of the fact that every database is structurally identical and share 
>> the "DataMap" resources.
>>
>> So, here are my questions:
>> 1. How do I use the Cayenne API to dynamically tie into the DBCP 
>> connection pool?
>> 2. Once I've done that, how do I dynamically switch schemas on a per 
>> HttpRequest basis?
>> 3. Will the solution work with multiple concurrent users each 
>> accessing their own database?
>>
>> In this message: 
>> http://article.gmane.org/gmane.comp.java.cayenne.user/10428/match=datasource 
>>
>> Andrus suggests a solution that I'm not sure will work in my case 
>> where every http request will require a schema change.  Has a 
>> solution been added to the API since this post?
>>
>> 4. Is Cayenne able to solve the memory consumption problem I'm having 
>> with Hibernate?  I.E., will Cayenne add little to no additional 
>> dedicated memory resources for every new DataDomain I create if I'm 
>> using a common database structure for each new database?
>>
>> Thank you for any help you can give me.
>>
>> Michael Martineau
>>
>>
>> Below is the test code I pieced together from several email posts:
>>
>> public class Main2
>> {
>>    public static void main(String[] args) throws SQLException
>>    {
>>        Main2 main = new Main2();
>>
>>        main.createNewDataDomain("db0");
>>        main.addUser("db0", "Michael");
>>
>>        main.createNewDataDomain("db1");
>>        main.addUser("db1", "Rebekah");
>>
>>        main.createNewDataDomain("db2");
>>        main.addUser("db2", "Andrew");
>>
>>        main.createNewDataDomain("db3");
>>        main.addUser("db3", "Rachael");
>>    }
>>
>>    private void createNewDataDomain(String name) throws SQLException
>>    {
>>        DataDomain templateDomain = 
>> Configuration.getSharedConfiguration().getDomain("Cayenne_DBX");
>>
>>        DataSource dataSource = new 
>> PoolManager("com.mysql.jdbc.Driver", 
>> "jdbc:mysql://localhost/cayenne_" + name, 1, 5, "cayenne", "cayenne");
>>
>>        DataNode dataNode = new DataNode("DynamicNode" + name);
>>        dataNode.setAdapter(new AutoAdapter(dataSource));
>>        dataNode.setDataSource(dataSource);
>>        
>> dataNode.addDataMap(templateDomain.getEntityResolver().getDataMap("Cayenne_DBXMap")); 
>>
>>
>>        DataDomain domainX = new DataDomain(name);
>>        domainX.setEntityResolver(templateDomain.getEntityResolver());
>>        domainX.addNode(dataNode);
>>
>>        Configuration c = Configuration.getSharedConfiguration();
>>        c.addDomain(domainX);
>>
>>    }
>>
>>    private void addUser(String domain, String username)
>>    {
>>        ObjectContext context = DataContext.createDataContext(domain);
>>
>>        User user = context.newObject(User.class);
>>        user.setName(username);
>>        context.commitChanges();
>>
>>        SelectQuery select1 = new SelectQuery(User.class);
>>        List<User> users = (List<User>)context.performQuery(select1);
>>
>>        for(User u : users)
>>        {
>>            System.out.println(u.getName());
>>        }
>>    }
>> }
>>
>>
>>
>>
>
>
>