Support multiple Derby systems in one VM

["David W. Van Couvering" <Da...@Sun.COM>]

I also wanted to share an idea to help move us towards the larger 
problem supporting multiple Derby systems within the same VM (currently 
that's not possible).

The idea is that we have a new public class called DerbySystem that's in 
it's own package.  This is basically a data source factory for a given 
Derby system, each with its own classloader.  So in the same VM I could 
do something like

// Note the properties could be loaded from a config file rather
// than hardcoded in the source
Properties propsA = new Properties();
propsA.setProperty("derby.system.home", "/path/to/system/a");
propsA.setProperty("derby.system.classpath",
"/home/derby/10.1/derby.jar");
DerbySystem systemA = new DerbySystem(propsA);

Properties propsB = new Properties();
propsB.setProperty("derby.system.home", "/path/to/system/b");
propsA.setProperty("derby.system.classpath", "/home/derby/10.2/derby.jar");
DerbySystem systemB = new DerbySystem(propsB);

DataSource dsA = systemA.getDataSource("jdbc:derby:wombat;create=true");
DataSource dsB = systemB.getDataSource("jdbc:derby:wombat;create=true");

Connection connA = dsA.getConnection();
Connection connB = dsB.getConnection();

Note that for this to really work we need to fix Derby so that there are 
no hardcoded uses of system properties, but instead all code uses a 
service to obtain configuration properties for a given system.

David

Re: Support multiple Derby systems in one VM

[Francois Orsini <fr...@gmail.com>]

I meant Java properties, NOT J2EE Env properties - as it would not really
make much sense at all - I mentioned Env as 'Environment' so that the Derby
DataSource Factory can decide of a classloader to use (whether an existing
one for this context (based on derby.system.home and
derby.system.classpathJava properties) or a new one to register - if
none (specific) registered,
then the default class loader would be used...

Yes, a prototype would be good to demonstrate this.

--francois

Re: Support multiple Derby systems in one VM

["David W. Van Couvering" <Da...@Sun.COM>]

I get your point that we don't want to have one classloader per data 
source, and we should use the default wherever possible.

I don't think we should depend on Env properties (I'm assuming you mean 
"java:comp/env" properties) because these are available only in J2EE 
environments.

A data source has properties on it, two of them could be 
derby.system.home and derby.system.classpath.  If derby.system.classpath 
is not set then we could just use the default classloader (or perhaps 
the one that prevents shadowing that I am considering) to load derby 
classes.

Just thoughts at this point.  All this needs to be demonstrated with 
working code...

David

Francois Orsini wrote:
> Agreed - The challenge in this example is to separate the 2 properties 
> set upon instancing appropriate datasources...Fundamentally, 2 separate 
> classloaders have to be created - Obviously we would not want to have to 
> create a separate classloader everytime we instantiate a new DataSource, 
> except if we have a way/mechanism to find out that we need to do so 
> (upon instantiating an EmbeddedDataSource object for instance)...Upon 
> instantiating a specialized DataSource in derby, I wonder if it'd be 
> possible to detect from the environment (i.e. Env Properties) if anyhing 
> relevant has changed that would lead Derby to return the proper 
> instantiated DataSource object based on the environment (Env properties) 
> - which means that the DataSource Factory would have to be smart enough 
> to figure this out (keeping track of the  various Env Properties) in 
> order to return the appropriate one (off a new or existing 
> classloader)...Now which properties ("derby.system.home", 
> "derby.system.classpath" - is that enough?) would be relevant enough to 
> figure out that this DataSource should be instantiated in a non-default 
> classloader but one that is either known or yet to be created...
> 
> Does this make any sense at all...?
> 
> --francois
> 
> On 11/23/05, *Daniel John Debrunner* < djd@debrunners.com 
> <ma...@debrunners.com>> wrote:
> 
> 
>     However, it does seem to me that JDBC already provides two factory
>     classes for getting connections, Driver and DataSource. Do we really
>     need another factory api?
> 
>     Dan.
> 
> 

Re: Support multiple Derby systems in one VM

[Francois Orsini <fr...@gmail.com>]

Agreed - The challenge in this example is to separate the 2 properties set
upon instancing appropriate datasources...Fundamentally, 2 separate
classloaders have to be created - Obviously we would not want to have to
create a separate classloader everytime we instantiate a new DataSource,
except if we have a way/mechanism to find out that we need to do so (upon
instantiating an EmbeddedDataSource object for instance)...Upon
instantiating a specialized DataSource in derby, I wonder if it'd be
possible to detect from the environment (i.e. Env Properties) if anyhing
relevant has changed that would lead Derby to return the proper instantiated
DataSource object based on the environment (Env properties) - which means
that the DataSource Factory would have to be smart enough to figure this out
(keeping track of the  various Env Properties) in order to return the
appropriate one (off a new or existing classloader)...Now which properties
("derby.system.home", "derby.system.classpath" - is that enough?) would be
relevant enough to figure out that this DataSource should be instantiated in
a non-default classloader but one that is either known or yet to be
created...

Does this make any sense at all...?

--francois

On 11/23/05, Daniel John Debrunner <djd@debrunners.com > wrote:
>
>
> However, it does seem to me that JDBC already provides two factory
> classes for getting connections, Driver and DataSource. Do we really
> need another factory api?
>
> Dan.
>
>

Re: Support multiple Derby systems in one VM

["David W. Van Couvering" <Da...@Sun.COM>]

I did want to say that although it's not my current itch to scratch, if 
anyone *does* decide to work on this bigger problem please let me know 
so we can collaborate and/or not step on each other's toes.

Thanks,

David

David W. Van Couvering wrote:
> Thanks for your comments.  If there is a way we could do this with the 
> existing factory APIs (using properties), that would be grand.  Perhaps 
> that's possible.
> 
> That said, my focus initially will be removing compatibility constraints 
> on common code; this would be a second phase, and one I am NOT working 
> on now if somebody else has this itch.
> 
> David
> 
> Daniel John Debrunner wrote:
> 
>> David W. Van Couvering wrote:
>>
>>
>>> I also wanted to share an idea to help move us towards the larger
>>> problem supporting multiple Derby systems within the same VM (currently
>>> that's not possible).
>>
>>
>>
>>> DataSource dsA = systemA.getDataSource("jdbc:derby:wombat;create=true");
>>> DataSource dsB = systemB.getDataSource("jdbc:derby:wombat;create=true");
>>>
>>> Connection connA = dsA.getConnection();
>>> Connection connB = dsB.getConnection();
>>
>>
>>
>> Has some potential, I do think we need to be careful about going in
>> directions that are not in line with existing ways to load JDBC drivers.
>> For example your code is mixing JDBC URLs with DataSource, that's not
>> expected JDBC programming. If the getDataSource() method just returned a
>> DataSource object, then this would be closer to expected JDBC 
>> programming.
>>
>> DataSource dsA = systemA.getDataSource();
>> DataSource dsB = systemB.getDataSource();
>>
>> // set JavaBean properties on dsA and dsB
>>
>> However, it does seem to me that JDBC already provides two factory
>> classes for getting connections, Driver and DataSource. Do we really
>> need another factory api?
>>
>> Dan.
>>

Re: Support multiple Derby systems in one VM

["David W. Van Couvering" <Da...@Sun.COM>]

Thanks for your comments.  If there is a way we could do this with the 
existing factory APIs (using properties), that would be grand.  Perhaps 
that's possible.

That said, my focus initially will be removing compatibility constraints 
on common code; this would be a second phase, and one I am NOT working 
on now if somebody else has this itch.

David

Daniel John Debrunner wrote:
> David W. Van Couvering wrote:
> 
> 
>>I also wanted to share an idea to help move us towards the larger
>>problem supporting multiple Derby systems within the same VM (currently
>>that's not possible).
> 
> 
>>DataSource dsA = systemA.getDataSource("jdbc:derby:wombat;create=true");
>>DataSource dsB = systemB.getDataSource("jdbc:derby:wombat;create=true");
>>
>>Connection connA = dsA.getConnection();
>>Connection connB = dsB.getConnection();
> 
> 
> Has some potential, I do think we need to be careful about going in
> directions that are not in line with existing ways to load JDBC drivers.
> For example your code is mixing JDBC URLs with DataSource, that's not
> expected JDBC programming. If the getDataSource() method just returned a
> DataSource object, then this would be closer to expected JDBC programming.
> 
> DataSource dsA = systemA.getDataSource();
> DataSource dsB = systemB.getDataSource();
> 
> // set JavaBean properties on dsA and dsB
> 
> However, it does seem to me that JDBC already provides two factory
> classes for getting connections, Driver and DataSource. Do we really
> need another factory api?
> 
> Dan.
> 

Re: Support multiple Derby systems in one VM

[Daniel John Debrunner <dj...@debrunners.com>]

David W. Van Couvering wrote:

> I also wanted to share an idea to help move us towards the larger
> problem supporting multiple Derby systems within the same VM (currently
> that's not possible).

> DataSource dsA = systemA.getDataSource("jdbc:derby:wombat;create=true");
> DataSource dsB = systemB.getDataSource("jdbc:derby:wombat;create=true");
> 
> Connection connA = dsA.getConnection();
> Connection connB = dsB.getConnection();

Has some potential, I do think we need to be careful about going in
directions that are not in line with existing ways to load JDBC drivers.
For example your code is mixing JDBC URLs with DataSource, that's not
expected JDBC programming. If the getDataSource() method just returned a
DataSource object, then this would be closer to expected JDBC programming.

DataSource dsA = systemA.getDataSource();
DataSource dsB = systemB.getDataSource();

// set JavaBean properties on dsA and dsB

However, it does seem to me that JDBC already provides two factory
classes for getting connections, Driver and DataSource. Do we really
need another factory api?

Dan.

Re: Support multiple Derby systems in one VM

[Jeremy Boynes <jb...@apache.org>]

This seems similar (at least in my head) to DataSourceFactory here:
http://mail-archives.apache.org/mod_mbox/db-derby-user/200511.mbox/browser

As Dan said, I think we need to be careful not to confuse the two
mechanisms for setting properties. However, I think there is a clear
separation between properties used to configure an engine and those used
to configure a datasource used to access the engine:

* properties on DerbySystem/DataSourceFactory that are used to configure
  the underlying engine itself (such as location of the error log)
* properties on the DataSource implementation that configure how the
  user is going to connect to that engine

In DataSourceFactory, the getFactory() method took a uri that identified
the desired implementation. The resulting factory could then be
configured with implementation specific properties (in Derby's case,
engine properties) before being used to create a DataSource. The
resulting DataSource can then be configured as usual.

For example,

DataSourceFactory systemA = DataSourceFactory.getFactory("derby:10.2");
DataSourceFactory systemB = DataSourceFactory.getFactory("derby:11.1");

systemA.setProperty("derby.system.home", "/home/dbA");
systemA.setProperty("derby.error.log", "/home/dbA/derby.log");

systemB.setProperty("derby.system.home", "/home/dbB");
systemB.setProperty("derby.error.log", "/home/dbB/foo.log");

DataSource dsA = systemA.getDataSource();
DataSource dsB = systemB.getDataSource();

BeanUtils.setProperty(dsA, "databaseName", "wombat");
Connection connA = dsA.getConnection();

BeanUtils.setProperty(dsB, "databaseName", "wombat");
Connection connB = dsB.getConnection();

--
Jeremy

David W. Van Couvering wrote:
> I also wanted to share an idea to help move us towards the larger
> problem supporting multiple Derby systems within the same VM (currently
> that's not possible).
> 
> The idea is that we have a new public class called DerbySystem that's in
> it's own package.  This is basically a data source factory for a given
> Derby system, each with its own classloader.  So in the same VM I could
> do something like
> 
> // Note the properties could be loaded from a config file rather
> // than hardcoded in the source
> Properties propsA = new Properties();
> propsA.setProperty("derby.system.home", "/path/to/system/a");
> propsA.setProperty("derby.system.classpath",
> "/home/derby/10.1/derby.jar");
> DerbySystem systemA = new DerbySystem(propsA);
> 
> Properties propsB = new Properties();
> propsB.setProperty("derby.system.home", "/path/to/system/b");
> propsA.setProperty("derby.system.classpath", "/home/derby/10.2/derby.jar");
> DerbySystem systemB = new DerbySystem(propsB);
> 
> DataSource dsA = systemA.getDataSource("jdbc:derby:wombat;create=true");
> DataSource dsB = systemB.getDataSource("jdbc:derby:wombat;create=true");
> 
> Connection connA = dsA.getConnection();
> Connection connB = dsB.getConnection();
> 
> Note that for this to really work we need to fix Derby so that there are
> no hardcoded uses of system properties, but instead all code uses a
> service to obtain configuration properties for a given system.
> 
> David