You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Jeremy Boynes <jb...@apache.org> on 2005/05/09 18:45:42 UTC

Unified DataSource API

There has been a fair amount of discussion recently about keeping 
compatibility between the embedded and client APIs for creating and 
configuring DataSources.

We currently have three DataSource implementations:
* EmbeddedDataSource - used by the SE embedded impl
* EmbeddedSimpleDataSource - used by the JSR169/ME impl
* ClientDataSource - used by the SE client code

not including the variations for Pooled and XA connnections, or ME 
support for the client.

IIRC the only one present in 10.0 was the EmbeddedDataSource so before 
we finalize 10.1 I think we should reexamine whether this is the way to go.

As an alternative I would suggest as single unified DataSource that 
works in conjunction with a connection-factory to create connections and 
a reference-factory to create References for binding into JNDI.

This would give us one public implementation of DataSource, simplifying 
the API for users and improving compatibility across environments. For 
example, the same application code could use either an embedded database 
or a client connection automatically based on the property values supplied.

--
Jeremy

Re: Unified DataSource API

Posted by David Van Couvering <Da...@Sun.COM>.
This sounds reasonable to me...

Jeremy Boynes wrote:

> There has been a fair amount of discussion recently about keeping 
> compatibility between the embedded and client APIs for creating and 
> configuring DataSources.
> 
> We currently have three DataSource implementations:
> * EmbeddedDataSource - used by the SE embedded impl
> * EmbeddedSimpleDataSource - used by the JSR169/ME impl
> * ClientDataSource - used by the SE client code
> 
> not including the variations for Pooled and XA connnections, or ME 
> support for the client.
> 
> IIRC the only one present in 10.0 was the EmbeddedDataSource so before 
> we finalize 10.1 I think we should reexamine whether this is the way to go.
> 
> As an alternative I would suggest as single unified DataSource that 
> works in conjunction with a connection-factory to create connections and 
> a reference-factory to create References for binding into JNDI.
> 
> This would give us one public implementation of DataSource, simplifying 
> the API for users and improving compatibility across environments. For 
> example, the same application code could use either an embedded database 
> or a client connection automatically based on the property values supplied.
> 
> -- 
> Jeremy

Re: Unified DataSource API

Posted by "Lance J. Andersen" <La...@Sun.COM>.

Jeremy Boynes wrote:

> Lance J. Andersen wrote:
>
>> The reality is that there are way too many applications which utilize 
>> java.sql.Driver implementations to depecrate this functionality.
>>
>> We are in JDBC 4 reducing the boiler plate code  by having 
>> DriverManager.getConnection() utilize the Server Provider Mechanism 
>> in J2SE to load the driver, removing the need for 
>> Class.forName(com.acme.FooDriver).
>>
>> There is no need for a J2EE application to utilize anything but a 
>> DataSource IMHO.
>>
>
> Agreed - especially as JNDI is required and the application should be 
> obtaining the DataSource from java:comp/env or in J2EE 5 via injection.
>
> There is still the J2SE environment. Originally Driver was the only 
> mechanism available and of course we still need to support it. 
> However, there are some issues with how they are configured:
>
> * the need to instantiate the implementation so that it can register
>   with the DriverManager.

  If you are using J2SE 6, you can make this mechanism work with any 
JDBC driver  if you follow the changes needed to make the service 
provider mechanism

>
> * the embedding of properties in the URL string providing no type
>   safety

agree

>
> * use of Properties, again providing no type safety
>
agee

> * a rather quirky API for getting property info from the Driver
>
> With the JDBC 2.0 extension and 3.0, DataSource became available as 
> the preferred mechanism. Ideally this would be via JNDI lookup 
> (although there is no standard provider and/or location for the 
> binding in J2SE), but as an alternative configuration can now be done 
> using standard JavaBean semantics (e.g. hard coding get/set calls, 
> serialization (.ser file), XML storage, reflection in conjunction with 
> PropertyEditors, etc.) This provides much more flexibility than 
> specialized Driver code.


The use DataSource from J2SE without JNDI,  just never caught on at 
least as of now.  Even do the docs state this is the preferred 
mechanism, the reality is that it is not used widely outside of  the 
container world right now

We had looked at the possibilities of trying to do further improvements 
in this area for JDBC 4, but due to the time contraints of  the J2SE 6 
schedule, had to remove this item.

If you have suggestions, I would love to hear them as I am working on a 
list of items for JDBC.next.

Please drop me a note off line Jeremy and we can discuss this further

Regards
Lance

>
>> Daniel John Debrunner wrote:
>>
>>>
>>> The current model means that the set of properties are all completely
>>> valid for a Derby datasource. Your proposed model changes that so that
>>> the valid set of properties is a subset based upon the value of one
>>> property ("network protocol"). How does that affect introspection 
>>> tools,
>>> does it make them easier or harder for the user? E.g. a generic tool
>>> will still display "port" and "serverName" if I select an embedded
>>> connection.
>>>
>
> The current model offers different combinations of properties 
> depending on the implementation class selected. The user must select 
> the right class for the type of connection they are using.
>
> The unified model allows the user to configure the properties as they 
> need and rely on the implementation to select the type of connection 
> appropriate based on the values they configure.
>
>>>
>>> So I guess I see your proposal only described in terms of a usage model
>>> that I believe is incorrect, namely application's direct use of Derby's
>>> DataSource implementations.
>>>
>
> I don't agree that it is incorrect - as described above I think the 
> use of JavaBean semantics makes end-user configuration easier.
>
>>> As a matter of interest, how does Geronimo re-create (store) a 
>>> DataSource?
>>>
>>>    - from a serialized object
>>>    - from a class name and a set of properties [key/value pairs] (e.g.
>>> in a config file)
>>>    - using Referencable
>>>
>
> DataSources are configured as outbound J2CA Connectors with the 
> property values initialized as ManagedConnectionFactory properties - 
> basically JavaBean configuration.
>
> One connector we have wraps a generic JDBC Driver so they can be 
> configured that way. We also have ones that wrap vendor specific 
> implementations, mapping MCF properties to DataSource properties.
>
> The app server always binds its own implementation of DataSource into 
> JNDI - this is typical and is as described in the JDBC spec. That 
> implementation delegates to the ConnectionManager that deals with 
> pooling and handle/component dis/re-association.
>
>>>
>>> So we end up with three data source implementations anyway :-)
>>> EmbeddedDataSource - existing api
>>
> and deprecated
>
>>> DerbyDataSource
>>
> the one used by applications
>
>>> ReferenceableDataSource
>>
> used where you have a J2SE environment using JNDI (which seems 
> atypical even though the spec says that is the preferred method; if 
> not would this not be handled by the appserver vendors rather than end 
> users?)
>
>>>
>>> At the bottom level you need some generic interface that hands a
>>> connection back, today it is based upon the stateless
>>> DriverManager/Driver model, with the connection being completely 
>>> defined
>>> by a JDBC URL and a set of properties.
>>>
>>> Not sure of the advantage to move to a DataSource model, because the
>>> DataSource model has state, the set of properties. So what state is
>>> required and how are these stateful objects managed?
>>>
>
> Well today there are two - one for the client and one for embedded.
>
> The state is there in both cases. In the current case it is held in 
> the URL/properties which is easy for the Driver impl but means the 
> DataSource impl is doing value->String conversion; the URL String and 
> properties are then parsed/validated by the factory.
>
> In the unified case it is held in a JavaBean which can be passed 
> directly to the factory (no parsing). The Driver impl still needs to 
> parse the URL/properties but there is no added overhead here.
>
> I don't see where state management comes in.
>
> -- 
> Jeremy


Re: Unified DataSource API

Posted by Jeremy Boynes <je...@boynes.com>.
Daniel John Debrunner wrote:
> 
> I wasn't talking about that DataSource, but a possible optimisation
> where a Java Bean (as you call it, I was using 'DataSource (like)' :-)

:-)

> could be pre-populated for the common connection requests, to avoid
> having to create a new Java Bean object each time. With such a scheme
> there would need to be management of those objects. Probably not worth
> it, but I wasn't exactly sure what you were proposing in this area, but
> your comments above clear that up.
> 

I get it. I agree its not worth it - the overhead of creating a new 
object and then initializing it is probably insignificant compared to 
the cost establishing the connection. If not, a simple cache in Driver 
from URL to DataSource-like JavaBean could be used.

--
Jeremy


Re: Unified DataSource API

Posted by Daniel John Debrunner <dj...@debrunners.com>.
Jeremy Boynes wrote:

> Daniel John Debrunner wrote:
> 
>>> I don't see where state management comes in.
>>
>>
>>
>> With your proposal when a JDBC URL request comes a DataSource (like)
>> object is required to obtain the connection, either:
>>
>>  - the object is created and filled in on the fly - possible extra
>> overhead, though maybe the existing properties object is similar
>> overhead today.
>>
> 
> This is what I am doing - the Driver implementation parses the
> URL/properties and constructs and initializes a JavaBean to hold them.
> This bean is passed to the connection factory which returns the
> Connection implementation.
> 
> In other words, what we've done is move the parsing code from the
> internal driver impl into the public driver impl.
> 
> When using a DataSource, it is already a JavaBean so can be passed down
> directly. This eliminates the parsing of the URL and String properties
> that is done today.

I was only talking about getting a connection via a JDBC URL.
> 
>>  - a object already exists with the correct properties - management
>> would be needed for these objects
>>
> 
> That would not be our issue - the DataSource state is managed by the
> JNDI implementation or whatever else is storing the connection properties.

I wasn't talking about that DataSource, but a possible optimisation
where a Java Bean (as you call it, I was using 'DataSource (like)' :-)
could be pre-populated for the common connection requests, to avoid
having to create a new Java Bean object each time. With such a scheme
there would need to be management of those objects. Probably not worth
it, but I wasn't exactly sure what you were proposing in this area, but
your comments above clear that up.

Dan.


Re: Unified DataSource API

Posted by Jeremy Boynes <jb...@apache.org>.
Daniel John Debrunner wrote:
>>I don't see where state management comes in.
> 
> 
> With your proposal when a JDBC URL request comes a DataSource (like)
> object is required to obtain the connection, either:
> 
>  - the object is created and filled in on the fly - possible extra
> overhead, though maybe the existing properties object is similar
> overhead today.
> 

This is what I am doing - the Driver implementation parses the 
URL/properties and constructs and initializes a JavaBean to hold them. 
This bean is passed to the connection factory which returns the 
Connection implementation.

In other words, what we've done is move the parsing code from the 
internal driver impl into the public driver impl.

When using a DataSource, it is already a JavaBean so can be passed down 
directly. This eliminates the parsing of the URL and String properties 
that is done today.

>  - a object already exists with the correct properties - management
> would be needed for these objects
> 

That would not be our issue - the DataSource state is managed by the 
JNDI implementation or whatever else is storing the connection properties.

> I think the implementation of how the connection is obtained under the
> covers could be separated from the issue of how many DataSource objects
> exist for Derby. Today it carries the state in properties object and
> possibly could be improved under the covers to use a DataSouce model.
> But we define the user visible DataSource model separately.

These are two separate issues - I just rolled them both into the code to 
show how it could be done.

--
Jeremy

Re: Unified DataSource API

Posted by Daniel John Debrunner <dj...@debrunners.com>.
Jeremy Boynes wrote:


>>> At the bottom level you need some generic interface that hands a
>>> connection back, today it is based upon the stateless
>>> DriverManager/Driver model, with the connection being completely defined
>>> by a JDBC URL and a set of properties.
>>>
>>> Not sure of the advantage to move to a DataSource model, because the
>>> DataSource model has state, the set of properties. So what state is
>>> required and how are these stateful objects managed?
>>>
> 
> Well today there are two - one for the client and one for embedded.
> 
> The state is there in both cases. In the current case it is held in the
> URL/properties which is easy for the Driver impl but means the
> DataSource impl is doing value->String conversion; the URL String and
> properties are then parsed/validated by the factory.
> 
> In the unified case it is held in a JavaBean which can be passed
> directly to the factory (no parsing). The Driver impl still needs to
> parse the URL/properties but there is no added overhead here.
> 
> I don't see where state management comes in.

With your proposal when a JDBC URL request comes a DataSource (like)
object is required to obtain the connection, either:

 - the object is created and filled in on the fly - possible extra
overhead, though maybe the existing properties object is similar
overhead today.

 - a object already exists with the correct properties - management
would be needed for these objects

I think the implementation of how the connection is obtained under the
covers could be separated from the issue of how many DataSource objects
exist for Derby. Today it carries the state in properties object and
possibly could be improved under the covers to use a DataSouce model.
But we define the user visible DataSource model separately.
Dan.


Re: Unified DataSource API

Posted by Jeremy Boynes <jb...@apache.org>.
Dibyendu Majumdar wrote:
> From: "Jeremy Boynes" <jb...@apache.org>
> 
>>The advantage of a unified model is that there is only one 
>>implementation *within Derby.* Having multiple ones for one product 
>>depending on the JVM and transport in use is just confusing.
> 
> 
> I think another problem with a unified implementation is that
> the various DataSource interfaces (DataSource, ConnectionPoolDataSource
> and XADataSource) are not related, and even have different signatures.
> 

Indeed, and one of the things that started this was that because of this 
their implementations should not extend the DataSource implementation 
(see the "DataSource class hierarchy" thread)

If you look at the code you'll see that these now extend a JavaBean base 
class that just defines the common properties but the specializations 
themselves are separate. What we have done is unify the client and 
embedded implementations of Pooled and XA.

--
Jeremy


Re: Unified DataSource API

Posted by Dibyendu Majumdar <di...@mazumdar.demon.co.uk>.
From: "Jeremy Boynes" <jb...@apache.org>
> The advantage of a unified model is that there is only one 
> implementation *within Derby.* Having multiple ones for one product 
> depending on the JVM and transport in use is just confusing.

I think another problem with a unified implementation is that
the various DataSource interfaces (DataSource, ConnectionPoolDataSource
and XADataSource) are not related, and even have different signatures.

Regards


Re: Unified DataSource API

Posted by Jeremy Boynes <jb...@apache.org>.
Dibyendu Majumdar wrote:
> From: "Jeremy Boynes" <jb...@apache.org>
> 
>>But all implementations follow the JavaBean model so it is fairly easy 
>>to write an application that can generically handle any vendor's 
>>implementation.
> 
> 
> Is that part of the spec, or just convention? Is it mandatory for a
> DataSource implementation to be also a JavaBean?
> 

JDBC spec, top of page 59.

--
Jeremy


Re: Unified DataSource API

Posted by Dibyendu Majumdar <di...@mazumdar.demon.co.uk>.
From: "Jeremy Boynes" <jb...@apache.org>
> But all implementations follow the JavaBean model so it is fairly easy 
> to write an application that can generically handle any vendor's 
> implementation.

Is that part of the spec, or just convention? Is it mandatory for a
DataSource implementation to be also a JavaBean?

Regards


Re: Unified DataSource API

Posted by Jeremy Boynes <jb...@apache.org>.
Dibyendu Majumdar wrote:
> 
> I think that building a unified DataSource API will actually not help
> very much because the problem is that there is no standard interface for a
> DataSource anyway. 

However, there is a common configuration model, that of a JavaBean. This 
is easy to map to a GUI for configuration, and easy to map to XML for 
configuration files (e.g. the way a ManagedConnectionFactory is configured).

> Even if Derby had a unified API, you still have to deal
> with
> differences in other DBMS implementations. So, in the end, the fact that
> Derby has different DataSource implementations is not disimilar to the
> situation where one has to deal with differences between the various
> DBMS vendors.

But all implementations follow the JavaBean model so it is fairly easy 
to write an application that can generically handle any vendor's 
implementation.

The advantage of a unified model is that there is only one 
implementation *within Derby.* Having multiple ones for one product 
depending on the JVM and transport in use is just confusing.

> 
> I have myself found it frustrating that there isn't a standard API
> for DataSources. It seems that the designers assumed you would always
> use JNDI, and did not really cater for J2SE or non-J2EE requirements.
> 

That was the assumption. Although it may not be ideal, I still think it 
is easier than the original DriverManager solution.

--
Jeremy


Re: Unified DataSource API

Posted by Dibyendu Majumdar <di...@mazumdar.demon.co.uk>.
From: "Jeremy Boynes" <jb...@apache.org>
> However, in some cases the application code will be creating the
> connection factory directly. I continue to believe this is easier using
> the JavaBean model with a DataSource implementation than with the legacy
> DriverManager model.

I think that building a unified DataSource API will actually not help
very much because the problem is that there is no standard interface for a
DataSource anyway. Even if Derby had a unified API, you still have to deal
with
differences in other DBMS implementations. So, in the end, the fact that
Derby has different DataSource implementations is not disimilar to the
situation where one has to deal with differences between the various
DBMS vendors.

I have myself found it frustrating that there isn't a standard API
for DataSources. It seems that the designers assumed you would always
use JNDI, and did not really cater for J2SE or non-J2EE requirements.

Regards



Re: Unified DataSource API

Posted by Jeremy Boynes <jb...@apache.org>.
Dibyendu Majumdar wrote:
> From: "Jeremy Boynes" <jb...@apache.org>
> 
>>With the JDBC 2.0 extension and 3.0, DataSource became available as the 
>>preferred mechanism. Ideally this would be via JNDI lookup (although 
>>there is no standard provider and/or location for the binding in J2SE), 
>>but as an alternative configuration can now be done using standard 
>>JavaBean semantics (e.g. hard coding get/set calls, serialization (.ser 
>>file), XML storage, reflection in conjunction with PropertyEditors, 
>>etc.) This provides much more flexibility than specialized Driver code.
> 
> 
> I think there is already a solution for this - to use a framework such as 
> Spring. DataSources can presumably be registered as beans, and injected
> into client code without any need to know what the implementation looks
> like.

Yes - in most cases (J2EE or with a framework like Spring) the user will 
  never actually see the underlying implementation and would only be 
using the DataSource interface. This is the usage model recommended in 
the spec.

However, in some cases the application code will be creating the 
connection factory directly. I continue to believe this is easier using 
the JavaBean model with a DataSource implementation than with the legacy 
DriverManager model.

--
Jeremy

Re: Unified DataSource API

Posted by Dibyendu Majumdar <di...@mazumdar.demon.co.uk>.
From: "Jeremy Boynes" <jb...@apache.org>
> With the JDBC 2.0 extension and 3.0, DataSource became available as the 
> preferred mechanism. Ideally this would be via JNDI lookup (although 
> there is no standard provider and/or location for the binding in J2SE), 
> but as an alternative configuration can now be done using standard 
> JavaBean semantics (e.g. hard coding get/set calls, serialization (.ser 
> file), XML storage, reflection in conjunction with PropertyEditors, 
> etc.) This provides much more flexibility than specialized Driver code.

I think there is already a solution for this - to use a framework such as 
Spring. DataSources can presumably be registered as beans, and injected
into client code without any need to know what the implementation looks
like.

Regards


Re: Unified DataSource API

Posted by Jeremy Boynes <jb...@apache.org>.
Lance J. Andersen wrote:
> The reality is that there are way too many applications which utilize 
> java.sql.Driver implementations to depecrate this functionality.
> 
> We are in JDBC 4 reducing the boiler plate code  by having 
> DriverManager.getConnection() utilize the Server Provider Mechanism in 
> J2SE to load the driver, removing the need for 
> Class.forName(com.acme.FooDriver).
> 
> There is no need for a J2EE application to utilize anything but a 
> DataSource IMHO.
> 

Agreed - especially as JNDI is required and the application should be 
obtaining the DataSource from java:comp/env or in J2EE 5 via injection.

There is still the J2SE environment. Originally Driver was the only 
mechanism available and of course we still need to support it. However, 
there are some issues with how they are configured:

* the need to instantiate the implementation so that it can register
   with the DriverManager.

* the embedding of properties in the URL string providing no type
   safety

* use of Properties, again providing no type safety

* a rather quirky API for getting property info from the Driver

With the JDBC 2.0 extension and 3.0, DataSource became available as the 
preferred mechanism. Ideally this would be via JNDI lookup (although 
there is no standard provider and/or location for the binding in J2SE), 
but as an alternative configuration can now be done using standard 
JavaBean semantics (e.g. hard coding get/set calls, serialization (.ser 
file), XML storage, reflection in conjunction with PropertyEditors, 
etc.) This provides much more flexibility than specialized Driver code.

> Daniel John Debrunner wrote:
> 
>>
>> The current model means that the set of properties are all completely
>> valid for a Derby datasource. Your proposed model changes that so that
>> the valid set of properties is a subset based upon the value of one
>> property ("network protocol"). How does that affect introspection tools,
>> does it make them easier or harder for the user? E.g. a generic tool
>> will still display "port" and "serverName" if I select an embedded
>> connection.
>>

The current model offers different combinations of properties depending 
on the implementation class selected. The user must select the right 
class for the type of connection they are using.

The unified model allows the user to configure the properties as they 
need and rely on the implementation to select the type of connection 
appropriate based on the values they configure.

>>
>> So I guess I see your proposal only described in terms of a usage model
>> that I believe is incorrect, namely application's direct use of Derby's
>> DataSource implementations.
>>

I don't agree that it is incorrect - as described above I think the use 
of JavaBean semantics makes end-user configuration easier.

>> As a matter of interest, how does Geronimo re-create (store) a 
>> DataSource?
>>
>>    - from a serialized object
>>    - from a class name and a set of properties [key/value pairs] (e.g.
>> in a config file)
>>    - using Referencable
>>

DataSources are configured as outbound J2CA Connectors with the property 
values initialized as ManagedConnectionFactory properties - basically 
JavaBean configuration.

One connector we have wraps a generic JDBC Driver so they can be 
configured that way. We also have ones that wrap vendor specific 
implementations, mapping MCF properties to DataSource properties.

The app server always binds its own implementation of DataSource into 
JNDI - this is typical and is as described in the JDBC spec. That 
implementation delegates to the ConnectionManager that deals with 
pooling and handle/component dis/re-association.

>>
>> So we end up with three data source implementations anyway :-)
>> EmbeddedDataSource - existing api
and deprecated

>> DerbyDataSource
the one used by applications

>> ReferenceableDataSource
used where you have a J2SE environment using JNDI (which seems atypical 
even though the spec says that is the preferred method; if not would 
this not be handled by the appserver vendors rather than end users?)

>>
>> At the bottom level you need some generic interface that hands a
>> connection back, today it is based upon the stateless
>> DriverManager/Driver model, with the connection being completely defined
>> by a JDBC URL and a set of properties.
>>
>> Not sure of the advantage to move to a DataSource model, because the
>> DataSource model has state, the set of properties. So what state is
>> required and how are these stateful objects managed?
>>

Well today there are two - one for the client and one for embedded.

The state is there in both cases. In the current case it is held in the 
URL/properties which is easy for the Driver impl but means the 
DataSource impl is doing value->String conversion; the URL String and 
properties are then parsed/validated by the factory.

In the unified case it is held in a JavaBean which can be passed 
directly to the factory (no parsing). The Driver impl still needs to 
parse the URL/properties but there is no added overhead here.

I don't see where state management comes in.

--
Jeremy

Re: Unified DataSource API

Posted by "Lance J. Andersen" <La...@Sun.COM>.
The reality is that there are way too many applications which utilize 
java.sql.Driver implementations to depecrate this functionality.

We are in JDBC 4 reducing the boiler plate code  by having 
DriverManager.getConnection() utilize the Server Provider Mechanism in 
J2SE to load the driver, removing the need for 
Class.forName(com.acme.FooDriver). 


There is no need for a J2EE application to utilize anything but a 
DataSource IMHO.

Regards
Lance

Daniel John Debrunner wrote:

>Jeremy Boynes wrote:
>
>  
>
>>The main changes are:
>>* DerbyDataSource is the implementation expected to be used by end
>>  user applications (both J2SE and J2ME). This implements the basic
>>  DataSource API returning a single physical connection.
>>    
>>
>
>Should we be encouraging J2SE/J2EE applications to use a DataSource
>*implementation*? JDBC 3.0 spec states:
>
><section 9.4.1>
>"DataSource properties are not intended to be directly accessible by
>JDBC clients. This design is reinforced by defining the access methods
>on the implementation class rather than on the public DataSource
>interface used by applications. Furthermore, the object that the client
>manipulates can be a wrapper that only implements the DataSource
>interface. The setter and getter methods for the properties need not be
>exposed to the client."
>
>Thus applications are only meant to deal with the javax.sql.DataSource
>and the JNDI lookup already provides database location independence.
>I believe J2SE applications are still intended to use the DriverManager
>interface and indeed the JDBC 4.0 expert group is improving the
>functionality of registering drivers using the driver manager, not
>deprecating it.
>
>Now JSR 169 for J2ME breaks the intent of section 9.4.1 and I complained
>about that to the JSR 169 expert group. I'm the reason that the examples
>in the JSR 169 DataSource javadoc are correct and show that reflection
>has to be used. [They used to show vendor specific setXXX methods being
>called on the javax.sql.DataSource interface].
>
>I'm not against a change like this, but I do want to understand how this
>improves anything, especially given how DataSource objects are meant to
>be used. One of the uses is for tools to use reflection to query the
>properties and provide gui access to them
>
><section 9.4.1>
>"Management tools that need to manipulate the properties of a DataSource
>implementation can access those properties using introspection."
>
>
>The current model means that the set of properties are all completely
>valid for a Derby datasource. Your proposed model changes that so that
>the valid set of properties is a subset based upon the value of one
>property ("network protocol"). How does that affect introspection tools,
>does it make them easier or harder for the user? E.g. a generic tool
>will still display "port" and "serverName" if I select an embedded
>connection.
>
>
>So I guess I see your proposal only described in terms of a usage model
>that I believe is incorrect, namely application's direct use of Derby's
>DataSource implementations.
>
>As a matter of interest, how does Geronimo re-create (store) a DataSource?
>
>    - from a serialized object
>    - from a class name and a set of properties [key/value pairs] (e.g.
>in a config file)
>    - using Referencable
>
>
>  
>
>>* ReferenceableDataSource is now a subclass rather than a base class.
>>  The reasoning is that it would only be used to bind data sources
>>  into a JNDI server that did not support Java Objects; this is a
>>  fairly specialized usecase. The only reason this is separate is
>>  because J2ME CDC/FP 1.0 does not support javax.naming.
>>    
>>
>
>So we end up with three data source implementations anyway :-)
>EmbeddedDataSource - existing api
>DerbyDataSource
>ReferenceableDataSource
>
>I would have loved to not have the special J2ME data source, and instead
>have EmbeddedDataSource work for both, but the fact that
>EmbeddedDataSource already implemented Referenceable meant I didn't want
>to change it to avoid backwards compatibility issues.
>
>  
>
>>* The intention is to use the same API for both client and server
>>  connections. The ConnectionFactory would create the appropriate
>>  Connection implementation depending on the properties in the
>>  DataSource. For example, if the networkProtocol is specified as
>>  "drda" and/or the serverName is non-null then a client connection
>>  would be created; if the protocol is "embedded" and/or serverName
>>  is null then we would create an embedded server. This makes the
>>  application code unaware of the physical location of the database.
>>  Yes, more concrete definition is needed for the semantics here.
>>
>>* The Driver implementation now uses the DataSource implementation
>>  (without using the actual interface so we can continue to support
>>  JDBC 2.0 core) rather than the other way around. This seemed like
>>  a natural refactoring given the approach adopted by JSR169.
>>    
>>
>
>At the bottom level you need some generic interface that hands a
>connection back, today it is based upon the stateless
>DriverManager/Driver model, with the connection being completely defined
>by a JDBC URL and a set of properties.
>
>Not sure of the advantage to move to a DataSource model, because the
>DataSource model has state, the set of properties. So what state is
>required and how are these stateful objects managed?
>
>Dan.
>
>  
>

Re: Unified DataSource API

Posted by Daniel John Debrunner <dj...@debrunners.com>.
Jeremy Boynes wrote:

> 
> The main changes are:
> * DerbyDataSource is the implementation expected to be used by end
>   user applications (both J2SE and J2ME). This implements the basic
>   DataSource API returning a single physical connection.

Should we be encouraging J2SE/J2EE applications to use a DataSource
*implementation*? JDBC 3.0 spec states:

<section 9.4.1>
"DataSource properties are not intended to be directly accessible by
JDBC clients. This design is reinforced by defining the access methods
on the implementation class rather than on the public DataSource
interface used by applications. Furthermore, the object that the client
manipulates can be a wrapper that only implements the DataSource
interface. The setter and getter methods for the properties need not be
exposed to the client."

Thus applications are only meant to deal with the javax.sql.DataSource
and the JNDI lookup already provides database location independence.
I believe J2SE applications are still intended to use the DriverManager
interface and indeed the JDBC 4.0 expert group is improving the
functionality of registering drivers using the driver manager, not
deprecating it.

Now JSR 169 for J2ME breaks the intent of section 9.4.1 and I complained
about that to the JSR 169 expert group. I'm the reason that the examples
in the JSR 169 DataSource javadoc are correct and show that reflection
has to be used. [They used to show vendor specific setXXX methods being
called on the javax.sql.DataSource interface].

I'm not against a change like this, but I do want to understand how this
improves anything, especially given how DataSource objects are meant to
be used. One of the uses is for tools to use reflection to query the
properties and provide gui access to them

<section 9.4.1>
"Management tools that need to manipulate the properties of a DataSource
implementation can access those properties using introspection."


The current model means that the set of properties are all completely
valid for a Derby datasource. Your proposed model changes that so that
the valid set of properties is a subset based upon the value of one
property ("network protocol"). How does that affect introspection tools,
does it make them easier or harder for the user? E.g. a generic tool
will still display "port" and "serverName" if I select an embedded
connection.


So I guess I see your proposal only described in terms of a usage model
that I believe is incorrect, namely application's direct use of Derby's
DataSource implementations.

As a matter of interest, how does Geronimo re-create (store) a DataSource?

    - from a serialized object
    - from a class name and a set of properties [key/value pairs] (e.g.
in a config file)
    - using Referencable


> * ReferenceableDataSource is now a subclass rather than a base class.
>   The reasoning is that it would only be used to bind data sources
>   into a JNDI server that did not support Java Objects; this is a
>   fairly specialized usecase. The only reason this is separate is
>   because J2ME CDC/FP 1.0 does not support javax.naming.

So we end up with three data source implementations anyway :-)
EmbeddedDataSource - existing api
DerbyDataSource
ReferenceableDataSource

I would have loved to not have the special J2ME data source, and instead
have EmbeddedDataSource work for both, but the fact that
EmbeddedDataSource already implemented Referenceable meant I didn't want
to change it to avoid backwards compatibility issues.

> * The intention is to use the same API for both client and server
>   connections. The ConnectionFactory would create the appropriate
>   Connection implementation depending on the properties in the
>   DataSource. For example, if the networkProtocol is specified as
>   "drda" and/or the serverName is non-null then a client connection
>   would be created; if the protocol is "embedded" and/or serverName
>   is null then we would create an embedded server. This makes the
>   application code unaware of the physical location of the database.
>   Yes, more concrete definition is needed for the semantics here.
> 
> * The Driver implementation now uses the DataSource implementation
>   (without using the actual interface so we can continue to support
>   JDBC 2.0 core) rather than the other way around. This seemed like
>   a natural refactoring given the approach adopted by JSR169.

At the bottom level you need some generic interface that hands a
connection back, today it is based upon the stateless
DriverManager/Driver model, with the connection being completely defined
by a JDBC URL and a set of properties.

Not sure of the advantage to move to a DataSource model, because the
DataSource model has state, the set of properties. So what state is
required and how are these stateful objects managed?

Dan.


Re: Unified DataSource API

Posted by Jeremy Boynes <jb...@apache.org>.
Daniel John Debrunner wrote:
> 
> It's an interesting idea but the details may make it not possible, for
> example References don't exist in J2ME.
> 
> Also since in the most case applications do not use Derby's or any
> DataSource implementation classes directly, I don't see how such a
> change would affect application code.
> 
> I'm not convinced that the api will end up being simpler, yes there
> would only be one class name for the Derby data source implementation,
> but now there will be more properties and properties that do not apply
> to certain connection types. E.g. the single data source would need to
> have the port and server properties, but these are not relevant in an
> embedded mode, so is the api simpler because of that?
> The code and api will also have to define and implement rules about how
> those properties behave, e.g. if conflicting properties are set, what
> happens.
> 
> A concrete proposal would be great, so we can see what is being made
> simpler, and how much simpler.
> 

Strawman committed into a branch - not all possible properties are 
included but I figure that's just a matter of detail.

The main changes are:
* DerbyDataSource is the implementation expected to be used by end
   user applications (both J2SE and J2ME). This implements the basic
   DataSource API returning a single physical connection.

* ReferenceableDataSource is now a subclass rather than a base class.
   The reasoning is that it would only be used to bind data sources
   into a JNDI server that did not support Java Objects; this is a
   fairly specialized usecase. The only reason this is separate is
   because J2ME CDC/FP 1.0 does not support javax.naming.

* The intention is to use the same API for both client and server
   connections. The ConnectionFactory would create the appropriate
   Connection implementation depending on the properties in the
   DataSource. For example, if the networkProtocol is specified as
   "drda" and/or the serverName is non-null then a client connection
   would be created; if the protocol is "embedded" and/or serverName
   is null then we would create an embedded server. This makes the
   application code unaware of the physical location of the database.
   Yes, more concrete definition is needed for the semantics here.

* The Driver implementation now uses the DataSource implementation
   (without using the actual interface so we can continue to support
   JDBC 2.0 core) rather than the other way around. This seemed like
   a natural refactoring given the approach adopted by JSR169.

--
Jeremy

Re: Unified DataSource API

Posted by Daniel John Debrunner <dj...@debrunners.com>.
Jeremy Boynes wrote:
> There has been a fair amount of discussion recently about keeping
> compatibility between the embedded and client APIs for creating and
> configuring DataSources.
> 
> We currently have three DataSource implementations:
> * EmbeddedDataSource - used by the SE embedded impl
> * EmbeddedSimpleDataSource - used by the JSR169/ME impl
> * ClientDataSource - used by the SE client code
> 
> not including the variations for Pooled and XA connnections, or ME
> support for the client.
> 
> IIRC the only one present in 10.0 was the EmbeddedDataSource so before
> we finalize 10.1 I think we should reexamine whether this is the way to go.
> 
> As an alternative I would suggest as single unified DataSource that
> works in conjunction with a connection-factory to create connections and
> a reference-factory to create References for binding into JNDI.
> 
> This would give us one public implementation of DataSource, simplifying
> the API for users and improving compatibility across environments. For
> example, the same application code could use either an embedded database
> or a client connection automatically based on the property values supplied.

It's an interesting idea but the details may make it not possible, for
example References don't exist in J2ME.

Also since in the most case applications do not use Derby's or any
DataSource implementation classes directly, I don't see how such a
change would affect application code.

I'm not convinced that the api will end up being simpler, yes there
would only be one class name for the Derby data source implementation,
but now there will be more properties and properties that do not apply
to certain connection types. E.g. the single data source would need to
have the port and server properties, but these are not relevant in an
embedded mode, so is the api simpler because of that?
The code and api will also have to define and implement rules about how
those properties behave, e.g. if conflicting properties are set, what
happens.

A concrete proposal would be great, so we can see what is being made
simpler, and how much simpler.

Dan.