You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cayenne.apache.org by Michael Martineau <mi...@familypursuit.com> on 2010/02/12 00:29:16 UTC
Using the Cayenne API to dynamically create DataDomains
Hello,
I'm evaluating to see if Cayenne would be a good replacement for
Hibernate in my current project. I'm creating a website using tomcat
where users register on the main website (e.g. www.mydomain.com). When
they register, they select their own sub-domain name for a private
database (e.g. myprivatedb.mydomain.com). The web app then creates a
new database instance for them using a common schema. Thus, each
database has the exact same structure. When each user goes to their own
sub-domain and signs in, they are connecting to their private database
(I use the sub-domain as a key to determine which database to use).
However, even though each user is accessing their own database, all
users are sharing the same JVM tomcat instance. Furthermore, all
databases are hosted by the same Mysql server instance.
After combing through the this email list I've been able to piece
together a simple example of using the Modeler to create a base
DataDomain/DataMap. Then using the Cayenne API, I've managed to
demonstrate to myself the ability to dynamically add new DataDomains
that use the DataMap template created using the Modeler (see code
below). However, in my example, not only am I creating new DataDomains,
I'm also creating a new connection pool for each DataDomain which is not
really what I want. I would like to use the commons-dbcp pool and have
all users share the same pool of connections to the database server,
even though they access different schemas (mysql databases).
I've managed to make this work in Hibernate by implementing their
ConnectionProvider interface. Each new database configuration
(SessionFactory) in Hibernate creates a ConnectionProvider which allowed
me to change the "default" schema on the connection (that I get from the
DBCP pool) just before handing it to Hibernate based on the website
sub-domain. The problem is that Hibernate must create a new
SessionFactory for every database schema. Unfortunately, this
"SessionFactory" takes up a fixed amount of memory (5MB in my case) on
the server for each new database regardless of whether the user who
created the private database is signed in and using the website or not.
What I need is something (Cayenne?) that will take advantage of the fact
that every database is structurally identical and share the "DataMap"
resources.
So, here are my questions:
1. How do I use the Cayenne API to dynamically tie into the DBCP
connection pool?
2. Once I've done that, how do I dynamically switch schemas on a per
HttpRequest basis?
3. Will the solution work with multiple concurrent users each accessing
their own database?
In this message:
http://article.gmane.org/gmane.comp.java.cayenne.user/10428/match=datasource
Andrus suggests a solution that I'm not sure will work in my case where
every http request will require a schema change. Has a solution been
added to the API since this post?
4. Is Cayenne able to solve the memory consumption problem I'm having
with Hibernate? I.E., will Cayenne add little to no additional
dedicated memory resources for every new DataDomain I create if I'm
using a common database structure for each new database?
Thank you for any help you can give me.
Michael Martineau
Below is the test code I pieced together from several email posts:
public class Main2
{
public static void main(String[] args) throws SQLException
{
Main2 main = new Main2();
main.createNewDataDomain("db0");
main.addUser("db0", "Michael");
main.createNewDataDomain("db1");
main.addUser("db1", "Rebekah");
main.createNewDataDomain("db2");
main.addUser("db2", "Andrew");
main.createNewDataDomain("db3");
main.addUser("db3", "Rachael");
}
private void createNewDataDomain(String name) throws SQLException
{
DataDomain templateDomain =
Configuration.getSharedConfiguration().getDomain("Cayenne_DBX");
DataSource dataSource = new
PoolManager("com.mysql.jdbc.Driver", "jdbc:mysql://localhost/cayenne_" +
name, 1, 5, "cayenne", "cayenne");
DataNode dataNode = new DataNode("DynamicNode" + name);
dataNode.setAdapter(new AutoAdapter(dataSource));
dataNode.setDataSource(dataSource);
dataNode.addDataMap(templateDomain.getEntityResolver().getDataMap("Cayenne_DBXMap"));
DataDomain domainX = new DataDomain(name);
domainX.setEntityResolver(templateDomain.getEntityResolver());
domainX.addNode(dataNode);
Configuration c = Configuration.getSharedConfiguration();
c.addDomain(domainX);
}
private void addUser(String domain, String username)
{
ObjectContext context = DataContext.createDataContext(domain);
User user = context.newObject(User.class);
user.setName(username);
context.commitChanges();
SelectQuery select1 = new SelectQuery(User.class);
List<User> users = (List<User>)context.performQuery(select1);
for(User u : users)
{
System.out.println(u.getName());
}
}
}
Re: Using the Cayenne API to dynamically create DataDomains
Posted by Michael Martineau <mi...@familypursuit.com>.
Andrus,
Thanks for the reply. For those interested and for future reference,
here is what I came up with. It seems to work just fine. The
"template" datasouce is setup using the Modeler to use DBCP. I'm new to
Cayenne, so this may not be the best way. If anyone with more experience
wants to comment, I would appreciate the feedback.
- Michael Martineau
public class Main4
{
public static void main(String[] args) throws SQLException
{
Main4 main = new Main4();
main.createNewDataDomain("db0");
main.addUser("db0", "Michael");
main.createNewDataDomain("db1");
main.addUser("db1", "Rebekah");
main.createNewDataDomain("db2");
main.addUser("db2", "Andrew");
main.createNewDataDomain("db3");
main.addUser("db3", "Rachael");
System.exit(0);
}
private void createNewDataDomain(String name) throws SQLException
{
String schema = "cayenne_" + name;
DataDomain templateDomain =
Configuration.getSharedConfiguration().getDomain("Cayenne_DBX");
DataSource dataSource =
templateDomain.getNode("Cayenne_DBXNode").getDataSource();
DataNode dataNode = new DataNode("DynamicNode" + name);
dataNode.setAdapter(new AutoAdapter(dataSource));
dataNode.setDataSource(dataSource);
MapLoader mapLoader = new MapLoader();
DataMap dataMap = mapLoader.loadDataMap("Cayenne_DBXMap.map.xml");
dataMap.setDefaultSchema(schema);
Map dbEntities = dataMap.getDbEntityMap();
for (Iterator entities = dbEntities.values().iterator();
entities.hasNext();)
{
DbEntity entity = (DbEntity) entities.next();
entity.setSchema(schema);
}
dataNode.addDataMap(dataMap);
DataDomain domainX = new DataDomain(name);
domainX.addNode(dataNode);
Configuration c = Configuration.getSharedConfiguration();
c.addDomain(domainX);
}
private void addUser(String domain, String username)
{
ObjectContext context = DataContext.createDataContext(domain);
User user = context.newObject(User.class);
user.setName(username);
context.commitChanges();
List<User> users = (List<User>) context.performQuery(new
SelectQuery(User.class));
for (User u : users)
{
System.out.println(u.getName());
}
}
}
On 2/12/2010 4:40 AM, Andrus Adamchik wrote:
> Hi Michael,
>
> Cayenne was initially designed to run in a multi-stack scenario like
> yours, so it is a good fit. Still since the scenario is not that
> common, there is no prepackaged recipe. But you are on the right
> track. See my comment below...
>
>> So, here are my questions:
>> 1. How do I use the Cayenne API to dynamically tie into the DBCP
>> connection pool?
>
> http://cayenne.apache.org/doc30/dbcpdatasourcefactory.html
>
> The above gives you a DBCP connection pool in your "template" domain.
> To share it between multiple domains, you need to reuse the DataSource
> attached to the DataNode of the template domain. So in your example
> you'd replace
>
>> DataSource dataSource = new PoolManager(...);
>
> with
>
> DataSource dataSource = templateDomain.getNode("XYZ").getDataSource();
>
> But that also means that you will need to clone the DataMap and reset
> the schema for all DbEntities to "cayenne_" + userName, instead of
> reusing them per your example below. (I actually suggest instead of
> cloning, re-loading DataMap from XML using MapLoader, which will be
> more reliable).
>
>> 2. Once I've done that, how do I dynamically switch schemas on a per
>> HttpRequest basis?
>
> Many ways to do that. E.g. via a servlet filter.
>
> void doFilter(..) {
> String userName = // get it from request somehow
> Configuration c = Configuration.getSharedConfiguration();
> DataDomain domain = c.getDomain(userName);
> if(domain == null) {
> // init on demand
> }
>
> BaseContext.bindThreadObjectContext(domain.createDataContext());
> }
>
> Later in your webapp code executed within request:
>
> ObjectContext context = BaseContext .getThreadObjectContext();
>
>> 3. Will the solution work with multiple concurrent users each
>> accessing their own database?
>
> Absolutely. The only code that may require some attention to thread
> safety is "// init on demand" line above.
>
>> 4. Is Cayenne able to solve the memory consumption problem I'm having
>> with Hibernate? I.E., will Cayenne add little to no additional
>> dedicated memory resources for every new DataDomain I create if I'm
>> using a common database structure for each new database?
>
> Extra Cayenne-related memory consumption per user will be roughly
> equal to the DataMap size. I suggest you put an upper cap on that,
> e.g. by storing user stacks not in shared Configuration, but rather in
> your own LRUMap, that will automatically kick out the stacks for
> inactive users. Another thing you may do is to reduce the cache size
> for each domain, by doing this:
>
> domain.getProperties().put("cayenne.DataRowStore.snapshot.size", "200");
>
> This will give you a fixed max memory profile.
>
> Hope this helps.
>
> Andrus
>
>
>
> On Feb 12, 2010, at 1:29 AM, Michael Martineau wrote:
>> Hello,
>>
>> I'm evaluating to see if Cayenne would be a good replacement for
>> Hibernate in my current project. I'm creating a website using tomcat
>> where users register on the main website (e.g. www.mydomain.com).
>> When they register, they select their own sub-domain name for a
>> private database (e.g. myprivatedb.mydomain.com). The web app then
>> creates a new database instance for them using a common schema.
>> Thus, each database has the exact same structure. When each user
>> goes to their own sub-domain and signs in, they are connecting to
>> their private database (I use the sub-domain as a key to determine
>> which database to use). However, even though each user is accessing
>> their own database, all users are sharing the same JVM tomcat
>> instance. Furthermore, all databases are hosted by the same Mysql
>> server instance.
>>
>> After combing through the this email list I've been able to piece
>> together a simple example of using the Modeler to create a base
>> DataDomain/DataMap. Then using the Cayenne API, I've managed to
>> demonstrate to myself the ability to dynamically add new DataDomains
>> that use the DataMap template created using the Modeler (see code
>> below). However, in my example, not only am I creating new
>> DataDomains, I'm also creating a new connection pool for each
>> DataDomain which is not really what I want. I would like to use the
>> commons-dbcp pool and have all users share the same pool of
>> connections to the database server, even though they access different
>> schemas (mysql databases).
>>
>> I've managed to make this work in Hibernate by implementing their
>> ConnectionProvider interface. Each new database configuration
>> (SessionFactory) in Hibernate creates a ConnectionProvider which
>> allowed me to change the "default" schema on the connection (that I
>> get from the DBCP pool) just before handing it to Hibernate based on
>> the website sub-domain. The problem is that Hibernate must create a
>> new SessionFactory for every database schema. Unfortunately, this
>> "SessionFactory" takes up a fixed amount of memory (5MB in my case)
>> on the server for each new database regardless of whether the user
>> who created the private database is signed in and using the website
>> or not. What I need is something (Cayenne?) that will take advantage
>> of the fact that every database is structurally identical and share
>> the "DataMap" resources.
>>
>> So, here are my questions:
>> 1. How do I use the Cayenne API to dynamically tie into the DBCP
>> connection pool?
>> 2. Once I've done that, how do I dynamically switch schemas on a per
>> HttpRequest basis?
>> 3. Will the solution work with multiple concurrent users each
>> accessing their own database?
>>
>> In this message:
>> http://article.gmane.org/gmane.comp.java.cayenne.user/10428/match=datasource
>>
>> Andrus suggests a solution that I'm not sure will work in my case
>> where every http request will require a schema change. Has a
>> solution been added to the API since this post?
>>
>> 4. Is Cayenne able to solve the memory consumption problem I'm having
>> with Hibernate? I.E., will Cayenne add little to no additional
>> dedicated memory resources for every new DataDomain I create if I'm
>> using a common database structure for each new database?
>>
>> Thank you for any help you can give me.
>>
>> Michael Martineau
>>
>>
>> Below is the test code I pieced together from several email posts:
>>
>> public class Main2
>> {
>> public static void main(String[] args) throws SQLException
>> {
>> Main2 main = new Main2();
>>
>> main.createNewDataDomain("db0");
>> main.addUser("db0", "Michael");
>>
>> main.createNewDataDomain("db1");
>> main.addUser("db1", "Rebekah");
>>
>> main.createNewDataDomain("db2");
>> main.addUser("db2", "Andrew");
>>
>> main.createNewDataDomain("db3");
>> main.addUser("db3", "Rachael");
>> }
>>
>> private void createNewDataDomain(String name) throws SQLException
>> {
>> DataDomain templateDomain =
>> Configuration.getSharedConfiguration().getDomain("Cayenne_DBX");
>>
>> DataSource dataSource = new
>> PoolManager("com.mysql.jdbc.Driver",
>> "jdbc:mysql://localhost/cayenne_" + name, 1, 5, "cayenne", "cayenne");
>>
>> DataNode dataNode = new DataNode("DynamicNode" + name);
>> dataNode.setAdapter(new AutoAdapter(dataSource));
>> dataNode.setDataSource(dataSource);
>>
>> dataNode.addDataMap(templateDomain.getEntityResolver().getDataMap("Cayenne_DBXMap"));
>>
>>
>> DataDomain domainX = new DataDomain(name);
>> domainX.setEntityResolver(templateDomain.getEntityResolver());
>> domainX.addNode(dataNode);
>>
>> Configuration c = Configuration.getSharedConfiguration();
>> c.addDomain(domainX);
>>
>> }
>>
>> private void addUser(String domain, String username)
>> {
>> ObjectContext context = DataContext.createDataContext(domain);
>>
>> User user = context.newObject(User.class);
>> user.setName(username);
>> context.commitChanges();
>>
>> SelectQuery select1 = new SelectQuery(User.class);
>> List<User> users = (List<User>)context.performQuery(select1);
>>
>> for(User u : users)
>> {
>> System.out.println(u.getName());
>> }
>> }
>> }
>>
>>
>>
>>
>
>
>
Re: Using the Cayenne API to dynamically create DataDomains
Posted by Andrus Adamchik <an...@objectstyle.org>.
Hi Michael,
Cayenne was initially designed to run in a multi-stack scenario like
yours, so it is a good fit. Still since the scenario is not that
common, there is no prepackaged recipe. But you are on the right
track. See my comment below...
> So, here are my questions:
> 1. How do I use the Cayenne API to dynamically tie into the DBCP
> connection pool?
http://cayenne.apache.org/doc30/dbcpdatasourcefactory.html
The above gives you a DBCP connection pool in your "template" domain.
To share it between multiple domains, you need to reuse the DataSource
attached to the DataNode of the template domain. So in your example
you'd replace
> DataSource dataSource = new PoolManager(...);
with
DataSource dataSource = templateDomain.getNode("XYZ").getDataSource();
But that also means that you will need to clone the DataMap and reset
the schema for all DbEntities to "cayenne_" + userName, instead of
reusing them per your example below. (I actually suggest instead of
cloning, re-loading DataMap from XML using MapLoader, which will be
more reliable).
> 2. Once I've done that, how do I dynamically switch schemas on a per
> HttpRequest basis?
Many ways to do that. E.g. via a servlet filter.
void doFilter(..) {
String userName = // get it from request somehow
Configuration c = Configuration.getSharedConfiguration();
DataDomain domain = c.getDomain(userName);
if(domain == null) {
// init on demand
}
BaseContext.bindThreadObjectContext(domain.createDataContext());
}
Later in your webapp code executed within request:
ObjectContext context = BaseContext .getThreadObjectContext();
> 3. Will the solution work with multiple concurrent users each
> accessing their own database?
Absolutely. The only code that may require some attention to thread
safety is "// init on demand" line above.
> 4. Is Cayenne able to solve the memory consumption problem I'm
> having with Hibernate? I.E., will Cayenne add little to no
> additional dedicated memory resources for every new DataDomain I
> create if I'm using a common database structure for each new database?
Extra Cayenne-related memory consumption per user will be roughly
equal to the DataMap size. I suggest you put an upper cap on that,
e.g. by storing user stacks not in shared Configuration, but rather in
your own LRUMap, that will automatically kick out the stacks for
inactive users. Another thing you may do is to reduce the cache size
for each domain, by doing this:
domain.getProperties().put("cayenne.DataRowStore.snapshot.size", "200");
This will give you a fixed max memory profile.
Hope this helps.
Andrus
On Feb 12, 2010, at 1:29 AM, Michael Martineau wrote:
> Hello,
>
> I'm evaluating to see if Cayenne would be a good replacement for
> Hibernate in my current project. I'm creating a website using
> tomcat where users register on the main website (e.g.
> www.mydomain.com). When they register, they select their own sub-
> domain name for a private database (e.g. myprivatedb.mydomain.com).
> The web app then creates a new database instance for them using a
> common schema. Thus, each database has the exact same structure.
> When each user goes to their own sub-domain and signs in, they are
> connecting to their private database (I use the sub-domain as a key
> to determine which database to use). However, even though each user
> is accessing their own database, all users are sharing the same JVM
> tomcat instance. Furthermore, all databases are hosted by the same
> Mysql server instance.
>
> After combing through the this email list I've been able to piece
> together a simple example of using the Modeler to create a base
> DataDomain/DataMap. Then using the Cayenne API, I've managed to
> demonstrate to myself the ability to dynamically add new DataDomains
> that use the DataMap template created using the Modeler (see code
> below). However, in my example, not only am I creating new
> DataDomains, I'm also creating a new connection pool for each
> DataDomain which is not really what I want. I would like to use the
> commons-dbcp pool and have all users share the same pool of
> connections to the database server, even though they access
> different schemas (mysql databases).
>
> I've managed to make this work in Hibernate by implementing their
> ConnectionProvider interface. Each new database configuration
> (SessionFactory) in Hibernate creates a ConnectionProvider which
> allowed me to change the "default" schema on the connection (that I
> get from the DBCP pool) just before handing it to Hibernate based on
> the website sub-domain. The problem is that Hibernate must create a
> new SessionFactory for every database schema. Unfortunately, this
> "SessionFactory" takes up a fixed amount of memory (5MB in my case)
> on the server for each new database regardless of whether the user
> who created the private database is signed in and using the website
> or not. What I need is something (Cayenne?) that will take
> advantage of the fact that every database is structurally identical
> and share the "DataMap" resources.
>
> So, here are my questions:
> 1. How do I use the Cayenne API to dynamically tie into the DBCP
> connection pool?
> 2. Once I've done that, how do I dynamically switch schemas on a per
> HttpRequest basis?
> 3. Will the solution work with multiple concurrent users each
> accessing their own database?
>
> In this message: http://article.gmane.org/gmane.comp.java.cayenne.user/10428/match=datasource
> Andrus suggests a solution that I'm not sure will work in my case
> where every http request will require a schema change. Has a
> solution been added to the API since this post?
>
> 4. Is Cayenne able to solve the memory consumption problem I'm
> having with Hibernate? I.E., will Cayenne add little to no
> additional dedicated memory resources for every new DataDomain I
> create if I'm using a common database structure for each new database?
>
> Thank you for any help you can give me.
>
> Michael Martineau
>
>
> Below is the test code I pieced together from several email posts:
>
> public class Main2
> {
> public static void main(String[] args) throws SQLException
> {
> Main2 main = new Main2();
>
> main.createNewDataDomain("db0");
> main.addUser("db0", "Michael");
>
> main.createNewDataDomain("db1");
> main.addUser("db1", "Rebekah");
>
> main.createNewDataDomain("db2");
> main.addUser("db2", "Andrew");
>
> main.createNewDataDomain("db3");
> main.addUser("db3", "Rachael");
> }
>
> private void createNewDataDomain(String name) throws SQLException
> {
> DataDomain templateDomain =
> Configuration.getSharedConfiguration().getDomain("Cayenne_DBX");
>
> DataSource dataSource = new
> PoolManager("com.mysql.jdbc.Driver", "jdbc:mysql://localhost/
> cayenne_" + name, 1, 5, "cayenne", "cayenne");
>
> DataNode dataNode = new DataNode("DynamicNode" + name);
> dataNode.setAdapter(new AutoAdapter(dataSource));
> dataNode.setDataSource(dataSource);
>
> dataNode
> .addDataMap
> (templateDomain.getEntityResolver().getDataMap("Cayenne_DBXMap"));
>
> DataDomain domainX = new DataDomain(name);
> domainX.setEntityResolver(templateDomain.getEntityResolver());
> domainX.addNode(dataNode);
>
> Configuration c = Configuration.getSharedConfiguration();
> c.addDomain(domainX);
>
> }
>
> private void addUser(String domain, String username)
> {
> ObjectContext context = DataContext.createDataContext(domain);
>
> User user = context.newObject(User.class);
> user.setName(username);
> context.commitChanges();
>
> SelectQuery select1 = new SelectQuery(User.class);
> List<User> users = (List<User>)context.performQuery(select1);
>
> for(User u : users)
> {
> System.out.println(u.getName());
> }
> }
> }
>
>
>
>