You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beehive.apache.org by Drew Varner <dr...@redops.org> on 2006/12/01 03:40:32 UTC
Idea for JDBC system control
I was playing around with an idea for the JDBC system control. The
ability to pass it a Connection object other than one declared in
it's annotations.
Here are a couple of use cases:
1) Integration with other persistence mechanisms. Let's say I have a
DAO that I can pass a Connection to. I can place the DAO and and JDBC
Control operations in the same transaction.
Connection conn = JDBCUtils.getConnection();
conn.setAutoCommit(false);
myJdbcControl.setExternalConnection(conn);
myJdbcControl.depositMoney(1000)
AccountAuditDAO.logDepositTransaction(1000,conn);
conn.commit();
conn.close();
2) Use it to put multiple JDBC Controls in the same db transaction
(same as above, replace the DAO with another JDBC control)
3) Make unit testing easier. You can pass a connection to a test
database into the control during testing and don't have to mess with
the JNDI setup for out-of-container testing.
4) Support for non-standard Connection sources. If you are using a
JDBC Control in a Struts app, you can grab the DataSource via the
Struts DataSource mechanism.
Setting an external connection would be manipulated by methods, not
annotations.
public void setExternalConnection(Connection conn);
public Connection getExternalConnection();
The control implementation would never close the external connection.
Would it be worth adding?
- Drew
Re: Idea for JDBC system control
Posted by Chad Schoettger <ch...@gmail.com>.
Hi Drew,
I may not have much time to work on this for the rest of the week
(Holidays rapidly approaching) so I'm going to go ahead and commit
your patch to add the setConnection() API to the JDBC control (minus
the lazy connection issue).
I think we are both in agreement that it would be very useful to be
able to configure a control's resource management. I am not convinced
that only changing how the JDBC control manages its resources is the
best long term approach.
I would prefer that we try to come up with an enhancement to the
controls framework so that any control could have its resource
management tuned by the user. I'll create a JIRA enhancement request
and we can start a new thread on the dev list about the best possible
solution for this issue.
- Chad
On 12/15/06, Drew Varner <dr...@redops.org> wrote:
> Hey Chad,
>
> I am trying to put my arms around what implementations we could break
> by moving to lazy connections and closing them when possible. There
> are four that come to mind. The first is a case where someone set's a
> transaction on a JDBC control via setAutoCommit(false).
>
> For example:
>
> myDatabaseControl.getConnection().setAutoCommit(false);
> myDatabaseControl.someTransactionalMethod();
> myDatabaseControl.getConnection().commit();
> myDatabaseControl.getConnection().setAutoCommit(true);
>
> This case is covered in unit tests
> (org.apache.beehive.controls.system.jdbc.units.results.TxResultsTest). I
> t's easy to preserve this case and my yet-to-be-submitted patch
> passes this test. We can't close out a connection after invoke() if
> the Connection is not in auto-commit mode. I provide a private flag
> inside the Control (boolean _connectionCloseable, defaults to true)
> to prevent this from happening. We still close out a connection if at
> release if it wasn't passed in via setConnection().
>
> The second case is where a user pulls a connection out of the Control
> to use elsewhere. For example:
>
> Connection conn = myDatabaseControl.getConnection();
> myDatabaseControl.someMethod(9);
> myDAO.deleteById(9,conn);
> conn.close();
>
> This case is not present in the unit tests. However, it is easy to
> solve this case. Once a user pulls the Connection out, we set
> _connectionCloseable to false. As the Javadoc for it suggests, this
> method is rarely called.
>
> The third case is where we return ResultSets, Iterators or RowSets.
> These are covered in the unit tests and my not-yet-submitted patch
> passes them. Flagging the connection as unsafe to close is easy.
> Calling ResultSetMapper's canCloseResultSet() method seems to be enough.
>
> The fourth case is not present in Beehive. It involves transaction
> annotations like the ones in WebLogic now. I have no way to test
> against these or know how to make this condition safe. Perhaps as
> Beehive uses Geronimo for tests we could evaluate this. Would a test
> for see if the connection is an XA connection be enough?
>
> I am going to setup a test case to measure the difference between the
> two implementations including:
>
> 1) Scalability with a small, fixed pool
> 2) Scalability with a small pool that can grow large
> 3) The number of file descriptors that are chewed up by each
> implementation
> 4) The behavior of the DataSource's connection pool
>
> If the difference is significant, perhaps behavior change could be
> could be considered for Beehive 2.x. If the difference is dramatic,
> perhaps we could evaluate a safe way to incorporate the behavior.
> It'd be nice to have an example of the cases my patch would break in
> unit tests.
>
> - Drew
>
>
> On Dec 15, 2006, at 6:21 PM, Chad Schoettger wrote:
>
> > Hi Drew,
> >
> > I'm in the process of applying your patch -- but there are a few
> > changes I'll need to make. Due to backwards compatibility issues I
> > don't think we can just switch the JDBC control to acquire a
> > connection in its invoke() and release it at the end of the invoke().
> > Existing applications may depend on the current behavior.
> >
> > Here are some options:
> >
> > 1) I've tweaked your patch so that it is now valid to have a JDBC
> > control without either a ConnectionDataSource or a ConnectionDriver
> > annotation.
> >
> > A JDBC control which does not include either of these annotations will
> > NOT try to acquire a connection when it receives an onAcquire event.
> > The connection must be set with the new 'setConnection()' api. This
> > should allow developers to have full control over connections being
> > used by the control.
> >
> > If the annotations are present the setConnection() api can still be
> > used to replace a connection the jdbc control has acquired.
> >
> >
> >
> > 2) It would still useful to introduce the concept of 'short-lived'
> > resources into the controls framework. I can imagine the scope being
> > that of a method call to the control (each method call causes an
> > onAcquire/onRelease event to be fired). I haven't given it a lot of
> > thought but it seems like we could add a property to the
> > @BaseProperites annotation for controls something like
> > 'methodResourceScope' which would trigger this behavior.
> >
> > @ControlInterface
> > @BaseProperites(methodResourceScope=true)
> > public interface MyCtrl {
> > .
> > .
> > .
> > }
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On 12/12/06, Drew Varner <dr...@redops.org> wrote:
> >> Ignore the "must setup JNDI if you don't need it" at the end, you
> >> must setup JNDI or explicitly null it out via a bean method.
> >>
> >> - Drew
> >>
> >> On Dec 13, 2006, at 1:28 AM, Drew Varner wrote:
> >>
> >> > Hey Chad,
> >> >
> >> > I was not sure where those events fired.
> >> >
> >> > My concern is picking up multiple connections where one is needed.
> >> > In an example where one database control is used the results are no
> >> > different than using any other persistence mechanism. However, if
> >> > we have many controls, it becomes problematic. If I use controls as
> >> > my basic building block for data abstraction, I could have one per
> >> > domain object. It's not beyond reason to expect 10 connections to
> >> > be used in a method. This is definitely the case where we wrap
> >> > multiple JDBC Controls in a custom control. Asking the architect to
> >> > accept a database connection pool five times or more than that of
> >> > another persistence solution seems steep. When load surges, you may
> >> > need one file descriptor per request to service the HTTP request
> >> > but ten to connect to the database. Manipulating 10 domain objects
> >> > in a call may seem extreme,
> >> >
> >> > We could work around this problem by creating one control per
> >> > database. This control would quickly become a maintenance monster.
> >> >
> >> > There are other applications for controls outside of short-lived
> >> > HTTP requests. BEA's WebLogic Integration product uses Controls
> >> > extensively in JPDs. I can't speak to the details of a JpdConext
> >> > but they don't sound like short lifespans. Assuming a rich data
> >> > model, the impact of a control keeping a tight grip on a Connection
> >> > could be high.
> >> >
> >> > I think we know whether or not it is safe to close the connection
> >> > after use. The ResultSetMapper's canCloseResultSet() should be
> >> > enough information. If you can close the ResultSet and the
> >> > Connection is in auto-commit mode, you should be able to close it
> >> > safely (according to the unit tests).
> >> >
> >> > While I appreciate that part of the controls mantra is
> >> > simplification when it comes to managing resources, DataSources
> >> > probably do a better job than a control of managing Connections
> >> > (running queries to identify stale connections, etc.). But, if we
> >> > limit controls to Pageflows or other web applications, these
> >> > features aren't a big deal.
> >> >
> >> > Having the database connection instantiated lazily makes testing
> >> > easier. Look at Chris' example of testing a Control with a JNDI
> >> > DataSource:
> >> >
> >> > http://dev2dev.bea.com/blog/hogue/archive/2006/07/
> >> > outofcontainer_1.html
> >> >
> >> > You have to reference the bean, not the interface. You have to
> >> > setup a JNDI Context if you use it in testing or not. If you lazily
> >> > instantiate, these hassles go away.
> >> >
> >> > - Drew
> >> >
> >> > On Dec 12, 2006, at 11:13 PM, Chad Schoettger wrote:
> >> >
> >> >> Hi Drew,
> >> >>
> >> >> I haven't had a chance to look at your patch yet (hopefully
> >> I'll have
> >> >> some time tomorrow) but I think the best solution may be to
> >> leave the
> >> >> aquisition/release of the connection in the onAcquire/onRelease
> >> >> methods.
> >> >>
> >> >> When using a JDBC control from a page flow these events provide a
> >> >> reasonably nice way of managing the connections. The connection is
> >> >> basically scoped for the page flow method the control is being
> >> >> invoked
> >> >> from:
> >> >>
> >> >> MyPageFlow.java
> >> >>
> >> >> @Control private JDBCControl _myJdbcControl;
> >> >>
> >> >> myAction() {
> >> >> _myJdbcControl.getRow() /* causes onAcquire() to be
> >> >> invoked */
> >> >> _myJdbcControl.getAnotherRow() /* does NOT cause onAquire
> >> () to
> >> >> be invoked */
> >> >> ......
> >> >> ......
> >> >> return .... /* causes onRelease() to be invoked() */
> >> >> }
> >> >>
> >> >> I really like the idea Eddie mentioned about having a set
> >> method for
> >> >> the connection -- if the user sets the connection, the jdbc
> >> control
> >> >> closes an open connection if it has one and uses the one
> >> provided by
> >> >> the user and basically noops in the onAcquire/onRelease
> >> methods. If
> >> >> the user sets the connection to null, the onAcquire/onRelease
> >> default
> >> >> behavior resumes.
> >> >>
> >> >> I do apologize if you were already aware of when onAcquire/
> >> onRelease
> >> >> were invoked, it seems to be an area which causes a fair amount of
> >> >> confusion.
> >> >>
> >> >> - Chad
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> On 12/11/06, Drew Varner <dr...@redops.org> wrote:
> >> >>> Ok, I now see why the connection cannot be closed immediately
> >> after
> >> >>> invoke: ResultSet results and possibly connected RowSets.
> >> >>>
> >> >>> Would it be possible to flag SQL methods to indicate that they
> >> don't
> >> >>> depend on an open connection? This option would not break
> >> existing
> >> >>> implementations.
> >> >>>
> >> >>> Another option would be to check the return type of the
> >> method. If
> >> >>> the return type is ResultSet or RowSet, we don't close it. I
> >> don't
> >> >>> like this solution as much. If a custom mapper does something
> >> weird
> >> >>> like wrapping a RowSet, we'll never know. This solution seems
> >> like
> >> >>> more of a hack.
> >> >>>
> >> >>> - Drew
> >> >>>
> >> >>>
> >> >>> On Dec 11, 2006, at 10:57 PM, Drew Varner wrote:
> >> >>>
> >> >>> > I attached the patch to BEEHIVE-1019 (http://issues.apache.org/
> >> >>> jira/
> >> >>> > browse/BEEHIVE-1019).
> >> >>> >
> >> >>> > The patch I attached moves getConnection() from onAcquire() to
> >> >>> > invoke().
> >> >>> >
> >> >>> > In the current version (grab connection when control is
> >> acquired)
> >> >>> > or the new version (if my patch is accepted) of the JDBC
> >> Control,
> >> >>> > the connection is pulled and held for quite a while. Imagine an
> >> >>> app
> >> >>> > has 10 database controls that point to the same database. If
> >> they
> >> >>> > are used in the same Pageflow, it eats up 10 connections to the
> >> >>> > database. That's a lot of overhead for one Pageflow to inflict
> >> >>> on a
> >> >>> > DataSource.
> >> >>> >
> >> >>> > Is there a reason not to move getConnection() at the
> >> beginning of
> >> >>> > invoke() and then close it out at the end of invoke()? This
> >> would
> >> >>> > be nice for JDBC controls with ConnectionDataSource
> >> >>> annotations. If
> >> >>> > there is some reason to hold on the connection like the current
> >> >>> > implementation, could we set up an annotation to allow this
> >> >>> > connection-conserving behavior?
> >> >>> >
> >> >>> > - Drew
> >> >>> >
> >> >>> > On Dec 1, 2006, at 10:07 AM, Eddie O'Neil wrote:
> >> >>> >
> >> >>> >> Drew--
> >> >>> >>
> >> >>> >> Definitely -- this would be a great feature that would help
> >> >>> with the
> >> >>> >> situations you described. Today, it's not possible to manage
> >> >>> the
> >> >>> >> Connection directly which makes it impossible to share it
> >> >>> between two
> >> >>> >> JdbcControl instances. It might be possible to just add a
> >> method
> >> >>> >> like:
> >> >>> >>
> >> >>> >> public void setConnection(Connection conn)
> >> >>> >>
> >> >>> >> that if called will short circuit the Connection creation code
> >> >>> in the
> >> >>> >> JdbcControl.
> >> >>> >>
> >> >>> >> This might be easy to add to the Control -- feel free to
> >> give it
> >> >>> >> a shot. :)
> >> >>> >>
> >> >>> >> Eddie
> >> >>> >>
> >> >>> >>
> >> >>> >> On 11/30/06, Drew Varner <dr...@redops.org> wrote:
> >> >>> >>> I was playing around with an idea for the JDBC system
> >> >>> control. The
> >> >>> >>> ability to pass it a Connection object other than one
> >> >>> declared in
> >> >>> >>> it's annotations.
> >> >>> >>>
> >> >>> >>> Here are a couple of use cases:
> >> >>> >>>
> >> >>> >>> 1) Integration with other persistence mechanisms. Let's say I
> >> >>> have a
> >> >>> >>> DAO that I can pass a Connection to. I can place the DAO
> >> and and
> >> >>> >>> JDBC
> >> >>> >>> Control operations in the same transaction.
> >> >>> >>>
> >> >>> >>> Connection conn = JDBCUtils.getConnection();
> >> >>> >>> conn.setAutoCommit(false);
> >> >>> >>> myJdbcControl.setExternalConnection(conn);
> >> >>> >>> myJdbcControl.depositMoney(1000)
> >> >>> >>> AccountAuditDAO.logDepositTransaction(1000,conn);
> >> >>> >>> conn.commit();
> >> >>> >>> conn.close();
> >> >>> >>>
> >> >>> >>> 2) Use it to put multiple JDBC Controls in the same db
> >> >>> transaction
> >> >>> >>> (same as above, replace the DAO with another JDBC control)
> >> >>> >>>
> >> >>> >>> 3) Make unit testing easier. You can pass a connection to
> >> a test
> >> >>> >>> database into the control during testing and don't have to
> >> >>> mess with
> >> >>> >>> the JNDI setup for out-of-container testing.
> >> >>> >>>
> >> >>> >>> 4) Support for non-standard Connection sources. If you are
> >> >>> using a
> >> >>> >>> JDBC Control in a Struts app, you can grab the DataSource via
> >> >>> the
> >> >>> >>> Struts DataSource mechanism.
> >> >>> >>>
> >> >>> >>> Setting an external connection would be manipulated by
> >> >>> methods, not
> >> >>> >>> annotations.
> >> >>> >>>
> >> >>> >>> public void setExternalConnection(Connection conn);
> >> >>> >>> public Connection getExternalConnection();
> >> >>> >>>
> >> >>> >>> The control implementation would never close the external
> >> >>> >>> connection.
> >> >>> >>>
> >> >>> >>> Would it be worth adding?
> >> >>> >>>
> >> >>> >>> - Drew
> >> >>> >>>
> >> >>> >>>
> >> >>> >>>
> >> >>> >
> >> >>>
> >> >>>
> >> >
> >>
> >>
>
>
Re: Idea for JDBC system control
Posted by Drew Varner <dr...@redops.org>.
Hey Chad,
I am trying to put my arms around what implementations we could break
by moving to lazy connections and closing them when possible. There
are four that come to mind. The first is a case where someone set's a
transaction on a JDBC control via setAutoCommit(false).
For example:
myDatabaseControl.getConnection().setAutoCommit(false);
myDatabaseControl.someTransactionalMethod();
myDatabaseControl.getConnection().commit();
myDatabaseControl.getConnection().setAutoCommit(true);
This case is covered in unit tests
(org.apache.beehive.controls.system.jdbc.units.results.TxResultsTest). I
t's easy to preserve this case and my yet-to-be-submitted patch
passes this test. We can't close out a connection after invoke() if
the Connection is not in auto-commit mode. I provide a private flag
inside the Control (boolean _connectionCloseable, defaults to true)
to prevent this from happening. We still close out a connection if at
release if it wasn't passed in via setConnection().
The second case is where a user pulls a connection out of the Control
to use elsewhere. For example:
Connection conn = myDatabaseControl.getConnection();
myDatabaseControl.someMethod(9);
myDAO.deleteById(9,conn);
conn.close();
This case is not present in the unit tests. However, it is easy to
solve this case. Once a user pulls the Connection out, we set
_connectionCloseable to false. As the Javadoc for it suggests, this
method is rarely called.
The third case is where we return ResultSets, Iterators or RowSets.
These are covered in the unit tests and my not-yet-submitted patch
passes them. Flagging the connection as unsafe to close is easy.
Calling ResultSetMapper's canCloseResultSet() method seems to be enough.
The fourth case is not present in Beehive. It involves transaction
annotations like the ones in WebLogic now. I have no way to test
against these or know how to make this condition safe. Perhaps as
Beehive uses Geronimo for tests we could evaluate this. Would a test
for see if the connection is an XA connection be enough?
I am going to setup a test case to measure the difference between the
two implementations including:
1) Scalability with a small, fixed pool
2) Scalability with a small pool that can grow large
3) The number of file descriptors that are chewed up by each
implementation
4) The behavior of the DataSource's connection pool
If the difference is significant, perhaps behavior change could be
could be considered for Beehive 2.x. If the difference is dramatic,
perhaps we could evaluate a safe way to incorporate the behavior.
It'd be nice to have an example of the cases my patch would break in
unit tests.
- Drew
On Dec 15, 2006, at 6:21 PM, Chad Schoettger wrote:
> Hi Drew,
>
> I'm in the process of applying your patch -- but there are a few
> changes I'll need to make. Due to backwards compatibility issues I
> don't think we can just switch the JDBC control to acquire a
> connection in its invoke() and release it at the end of the invoke().
> Existing applications may depend on the current behavior.
>
> Here are some options:
>
> 1) I've tweaked your patch so that it is now valid to have a JDBC
> control without either a ConnectionDataSource or a ConnectionDriver
> annotation.
>
> A JDBC control which does not include either of these annotations will
> NOT try to acquire a connection when it receives an onAcquire event.
> The connection must be set with the new 'setConnection()' api. This
> should allow developers to have full control over connections being
> used by the control.
>
> If the annotations are present the setConnection() api can still be
> used to replace a connection the jdbc control has acquired.
>
>
>
> 2) It would still useful to introduce the concept of 'short-lived'
> resources into the controls framework. I can imagine the scope being
> that of a method call to the control (each method call causes an
> onAcquire/onRelease event to be fired). I haven't given it a lot of
> thought but it seems like we could add a property to the
> @BaseProperites annotation for controls something like
> 'methodResourceScope' which would trigger this behavior.
>
> @ControlInterface
> @BaseProperites(methodResourceScope=true)
> public interface MyCtrl {
> .
> .
> .
> }
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On 12/12/06, Drew Varner <dr...@redops.org> wrote:
>> Ignore the "must setup JNDI if you don't need it" at the end, you
>> must setup JNDI or explicitly null it out via a bean method.
>>
>> - Drew
>>
>> On Dec 13, 2006, at 1:28 AM, Drew Varner wrote:
>>
>> > Hey Chad,
>> >
>> > I was not sure where those events fired.
>> >
>> > My concern is picking up multiple connections where one is needed.
>> > In an example where one database control is used the results are no
>> > different than using any other persistence mechanism. However, if
>> > we have many controls, it becomes problematic. If I use controls as
>> > my basic building block for data abstraction, I could have one per
>> > domain object. It's not beyond reason to expect 10 connections to
>> > be used in a method. This is definitely the case where we wrap
>> > multiple JDBC Controls in a custom control. Asking the architect to
>> > accept a database connection pool five times or more than that of
>> > another persistence solution seems steep. When load surges, you may
>> > need one file descriptor per request to service the HTTP request
>> > but ten to connect to the database. Manipulating 10 domain objects
>> > in a call may seem extreme,
>> >
>> > We could work around this problem by creating one control per
>> > database. This control would quickly become a maintenance monster.
>> >
>> > There are other applications for controls outside of short-lived
>> > HTTP requests. BEA's WebLogic Integration product uses Controls
>> > extensively in JPDs. I can't speak to the details of a JpdConext
>> > but they don't sound like short lifespans. Assuming a rich data
>> > model, the impact of a control keeping a tight grip on a Connection
>> > could be high.
>> >
>> > I think we know whether or not it is safe to close the connection
>> > after use. The ResultSetMapper's canCloseResultSet() should be
>> > enough information. If you can close the ResultSet and the
>> > Connection is in auto-commit mode, you should be able to close it
>> > safely (according to the unit tests).
>> >
>> > While I appreciate that part of the controls mantra is
>> > simplification when it comes to managing resources, DataSources
>> > probably do a better job than a control of managing Connections
>> > (running queries to identify stale connections, etc.). But, if we
>> > limit controls to Pageflows or other web applications, these
>> > features aren't a big deal.
>> >
>> > Having the database connection instantiated lazily makes testing
>> > easier. Look at Chris' example of testing a Control with a JNDI
>> > DataSource:
>> >
>> > http://dev2dev.bea.com/blog/hogue/archive/2006/07/
>> > outofcontainer_1.html
>> >
>> > You have to reference the bean, not the interface. You have to
>> > setup a JNDI Context if you use it in testing or not. If you lazily
>> > instantiate, these hassles go away.
>> >
>> > - Drew
>> >
>> > On Dec 12, 2006, at 11:13 PM, Chad Schoettger wrote:
>> >
>> >> Hi Drew,
>> >>
>> >> I haven't had a chance to look at your patch yet (hopefully
>> I'll have
>> >> some time tomorrow) but I think the best solution may be to
>> leave the
>> >> aquisition/release of the connection in the onAcquire/onRelease
>> >> methods.
>> >>
>> >> When using a JDBC control from a page flow these events provide a
>> >> reasonably nice way of managing the connections. The connection is
>> >> basically scoped for the page flow method the control is being
>> >> invoked
>> >> from:
>> >>
>> >> MyPageFlow.java
>> >>
>> >> @Control private JDBCControl _myJdbcControl;
>> >>
>> >> myAction() {
>> >> _myJdbcControl.getRow() /* causes onAcquire() to be
>> >> invoked */
>> >> _myJdbcControl.getAnotherRow() /* does NOT cause onAquire
>> () to
>> >> be invoked */
>> >> ......
>> >> ......
>> >> return .... /* causes onRelease() to be invoked() */
>> >> }
>> >>
>> >> I really like the idea Eddie mentioned about having a set
>> method for
>> >> the connection -- if the user sets the connection, the jdbc
>> control
>> >> closes an open connection if it has one and uses the one
>> provided by
>> >> the user and basically noops in the onAcquire/onRelease
>> methods. If
>> >> the user sets the connection to null, the onAcquire/onRelease
>> default
>> >> behavior resumes.
>> >>
>> >> I do apologize if you were already aware of when onAcquire/
>> onRelease
>> >> were invoked, it seems to be an area which causes a fair amount of
>> >> confusion.
>> >>
>> >> - Chad
>> >>
>> >>
>> >>
>> >>
>> >> On 12/11/06, Drew Varner <dr...@redops.org> wrote:
>> >>> Ok, I now see why the connection cannot be closed immediately
>> after
>> >>> invoke: ResultSet results and possibly connected RowSets.
>> >>>
>> >>> Would it be possible to flag SQL methods to indicate that they
>> don't
>> >>> depend on an open connection? This option would not break
>> existing
>> >>> implementations.
>> >>>
>> >>> Another option would be to check the return type of the
>> method. If
>> >>> the return type is ResultSet or RowSet, we don't close it. I
>> don't
>> >>> like this solution as much. If a custom mapper does something
>> weird
>> >>> like wrapping a RowSet, we'll never know. This solution seems
>> like
>> >>> more of a hack.
>> >>>
>> >>> - Drew
>> >>>
>> >>>
>> >>> On Dec 11, 2006, at 10:57 PM, Drew Varner wrote:
>> >>>
>> >>> > I attached the patch to BEEHIVE-1019 (http://issues.apache.org/
>> >>> jira/
>> >>> > browse/BEEHIVE-1019).
>> >>> >
>> >>> > The patch I attached moves getConnection() from onAcquire() to
>> >>> > invoke().
>> >>> >
>> >>> > In the current version (grab connection when control is
>> acquired)
>> >>> > or the new version (if my patch is accepted) of the JDBC
>> Control,
>> >>> > the connection is pulled and held for quite a while. Imagine an
>> >>> app
>> >>> > has 10 database controls that point to the same database. If
>> they
>> >>> > are used in the same Pageflow, it eats up 10 connections to the
>> >>> > database. That's a lot of overhead for one Pageflow to inflict
>> >>> on a
>> >>> > DataSource.
>> >>> >
>> >>> > Is there a reason not to move getConnection() at the
>> beginning of
>> >>> > invoke() and then close it out at the end of invoke()? This
>> would
>> >>> > be nice for JDBC controls with ConnectionDataSource
>> >>> annotations. If
>> >>> > there is some reason to hold on the connection like the current
>> >>> > implementation, could we set up an annotation to allow this
>> >>> > connection-conserving behavior?
>> >>> >
>> >>> > - Drew
>> >>> >
>> >>> > On Dec 1, 2006, at 10:07 AM, Eddie O'Neil wrote:
>> >>> >
>> >>> >> Drew--
>> >>> >>
>> >>> >> Definitely -- this would be a great feature that would help
>> >>> with the
>> >>> >> situations you described. Today, it's not possible to manage
>> >>> the
>> >>> >> Connection directly which makes it impossible to share it
>> >>> between two
>> >>> >> JdbcControl instances. It might be possible to just add a
>> method
>> >>> >> like:
>> >>> >>
>> >>> >> public void setConnection(Connection conn)
>> >>> >>
>> >>> >> that if called will short circuit the Connection creation code
>> >>> in the
>> >>> >> JdbcControl.
>> >>> >>
>> >>> >> This might be easy to add to the Control -- feel free to
>> give it
>> >>> >> a shot. :)
>> >>> >>
>> >>> >> Eddie
>> >>> >>
>> >>> >>
>> >>> >> On 11/30/06, Drew Varner <dr...@redops.org> wrote:
>> >>> >>> I was playing around with an idea for the JDBC system
>> >>> control. The
>> >>> >>> ability to pass it a Connection object other than one
>> >>> declared in
>> >>> >>> it's annotations.
>> >>> >>>
>> >>> >>> Here are a couple of use cases:
>> >>> >>>
>> >>> >>> 1) Integration with other persistence mechanisms. Let's say I
>> >>> have a
>> >>> >>> DAO that I can pass a Connection to. I can place the DAO
>> and and
>> >>> >>> JDBC
>> >>> >>> Control operations in the same transaction.
>> >>> >>>
>> >>> >>> Connection conn = JDBCUtils.getConnection();
>> >>> >>> conn.setAutoCommit(false);
>> >>> >>> myJdbcControl.setExternalConnection(conn);
>> >>> >>> myJdbcControl.depositMoney(1000)
>> >>> >>> AccountAuditDAO.logDepositTransaction(1000,conn);
>> >>> >>> conn.commit();
>> >>> >>> conn.close();
>> >>> >>>
>> >>> >>> 2) Use it to put multiple JDBC Controls in the same db
>> >>> transaction
>> >>> >>> (same as above, replace the DAO with another JDBC control)
>> >>> >>>
>> >>> >>> 3) Make unit testing easier. You can pass a connection to
>> a test
>> >>> >>> database into the control during testing and don't have to
>> >>> mess with
>> >>> >>> the JNDI setup for out-of-container testing.
>> >>> >>>
>> >>> >>> 4) Support for non-standard Connection sources. If you are
>> >>> using a
>> >>> >>> JDBC Control in a Struts app, you can grab the DataSource via
>> >>> the
>> >>> >>> Struts DataSource mechanism.
>> >>> >>>
>> >>> >>> Setting an external connection would be manipulated by
>> >>> methods, not
>> >>> >>> annotations.
>> >>> >>>
>> >>> >>> public void setExternalConnection(Connection conn);
>> >>> >>> public Connection getExternalConnection();
>> >>> >>>
>> >>> >>> The control implementation would never close the external
>> >>> >>> connection.
>> >>> >>>
>> >>> >>> Would it be worth adding?
>> >>> >>>
>> >>> >>> - Drew
>> >>> >>>
>> >>> >>>
>> >>> >>>
>> >>> >
>> >>>
>> >>>
>> >
>>
>>
Re: Idea for JDBC system control
Posted by Chad Schoettger <ch...@gmail.com>.
Hi Drew,
I'm in the process of applying your patch -- but there are a few
changes I'll need to make. Due to backwards compatibility issues I
don't think we can just switch the JDBC control to acquire a
connection in its invoke() and release it at the end of the invoke().
Existing applications may depend on the current behavior.
Here are some options:
1) I've tweaked your patch so that it is now valid to have a JDBC
control without either a ConnectionDataSource or a ConnectionDriver
annotation.
A JDBC control which does not include either of these annotations will
NOT try to acquire a connection when it receives an onAcquire event.
The connection must be set with the new 'setConnection()' api. This
should allow developers to have full control over connections being
used by the control.
If the annotations are present the setConnection() api can still be
used to replace a connection the jdbc control has acquired.
2) It would still useful to introduce the concept of 'short-lived'
resources into the controls framework. I can imagine the scope being
that of a method call to the control (each method call causes an
onAcquire/onRelease event to be fired). I haven't given it a lot of
thought but it seems like we could add a property to the
@BaseProperites annotation for controls something like
'methodResourceScope' which would trigger this behavior.
@ControlInterface
@BaseProperites(methodResourceScope=true)
public interface MyCtrl {
.
.
.
}
On 12/12/06, Drew Varner <dr...@redops.org> wrote:
> Ignore the "must setup JNDI if you don't need it" at the end, you
> must setup JNDI or explicitly null it out via a bean method.
>
> - Drew
>
> On Dec 13, 2006, at 1:28 AM, Drew Varner wrote:
>
> > Hey Chad,
> >
> > I was not sure where those events fired.
> >
> > My concern is picking up multiple connections where one is needed.
> > In an example where one database control is used the results are no
> > different than using any other persistence mechanism. However, if
> > we have many controls, it becomes problematic. If I use controls as
> > my basic building block for data abstraction, I could have one per
> > domain object. It's not beyond reason to expect 10 connections to
> > be used in a method. This is definitely the case where we wrap
> > multiple JDBC Controls in a custom control. Asking the architect to
> > accept a database connection pool five times or more than that of
> > another persistence solution seems steep. When load surges, you may
> > need one file descriptor per request to service the HTTP request
> > but ten to connect to the database. Manipulating 10 domain objects
> > in a call may seem extreme,
> >
> > We could work around this problem by creating one control per
> > database. This control would quickly become a maintenance monster.
> >
> > There are other applications for controls outside of short-lived
> > HTTP requests. BEA's WebLogic Integration product uses Controls
> > extensively in JPDs. I can't speak to the details of a JpdConext
> > but they don't sound like short lifespans. Assuming a rich data
> > model, the impact of a control keeping a tight grip on a Connection
> > could be high.
> >
> > I think we know whether or not it is safe to close the connection
> > after use. The ResultSetMapper's canCloseResultSet() should be
> > enough information. If you can close the ResultSet and the
> > Connection is in auto-commit mode, you should be able to close it
> > safely (according to the unit tests).
> >
> > While I appreciate that part of the controls mantra is
> > simplification when it comes to managing resources, DataSources
> > probably do a better job than a control of managing Connections
> > (running queries to identify stale connections, etc.). But, if we
> > limit controls to Pageflows or other web applications, these
> > features aren't a big deal.
> >
> > Having the database connection instantiated lazily makes testing
> > easier. Look at Chris' example of testing a Control with a JNDI
> > DataSource:
> >
> > http://dev2dev.bea.com/blog/hogue/archive/2006/07/
> > outofcontainer_1.html
> >
> > You have to reference the bean, not the interface. You have to
> > setup a JNDI Context if you use it in testing or not. If you lazily
> > instantiate, these hassles go away.
> >
> > - Drew
> >
> > On Dec 12, 2006, at 11:13 PM, Chad Schoettger wrote:
> >
> >> Hi Drew,
> >>
> >> I haven't had a chance to look at your patch yet (hopefully I'll have
> >> some time tomorrow) but I think the best solution may be to leave the
> >> aquisition/release of the connection in the onAcquire/onRelease
> >> methods.
> >>
> >> When using a JDBC control from a page flow these events provide a
> >> reasonably nice way of managing the connections. The connection is
> >> basically scoped for the page flow method the control is being
> >> invoked
> >> from:
> >>
> >> MyPageFlow.java
> >>
> >> @Control private JDBCControl _myJdbcControl;
> >>
> >> myAction() {
> >> _myJdbcControl.getRow() /* causes onAcquire() to be
> >> invoked */
> >> _myJdbcControl.getAnotherRow() /* does NOT cause onAquire() to
> >> be invoked */
> >> ......
> >> ......
> >> return .... /* causes onRelease() to be invoked() */
> >> }
> >>
> >> I really like the idea Eddie mentioned about having a set method for
> >> the connection -- if the user sets the connection, the jdbc control
> >> closes an open connection if it has one and uses the one provided by
> >> the user and basically noops in the onAcquire/onRelease methods. If
> >> the user sets the connection to null, the onAcquire/onRelease default
> >> behavior resumes.
> >>
> >> I do apologize if you were already aware of when onAcquire/onRelease
> >> were invoked, it seems to be an area which causes a fair amount of
> >> confusion.
> >>
> >> - Chad
> >>
> >>
> >>
> >>
> >> On 12/11/06, Drew Varner <dr...@redops.org> wrote:
> >>> Ok, I now see why the connection cannot be closed immediately after
> >>> invoke: ResultSet results and possibly connected RowSets.
> >>>
> >>> Would it be possible to flag SQL methods to indicate that they don't
> >>> depend on an open connection? This option would not break existing
> >>> implementations.
> >>>
> >>> Another option would be to check the return type of the method. If
> >>> the return type is ResultSet or RowSet, we don't close it. I don't
> >>> like this solution as much. If a custom mapper does something weird
> >>> like wrapping a RowSet, we'll never know. This solution seems like
> >>> more of a hack.
> >>>
> >>> - Drew
> >>>
> >>>
> >>> On Dec 11, 2006, at 10:57 PM, Drew Varner wrote:
> >>>
> >>> > I attached the patch to BEEHIVE-1019 (http://issues.apache.org/
> >>> jira/
> >>> > browse/BEEHIVE-1019).
> >>> >
> >>> > The patch I attached moves getConnection() from onAcquire() to
> >>> > invoke().
> >>> >
> >>> > In the current version (grab connection when control is acquired)
> >>> > or the new version (if my patch is accepted) of the JDBC Control,
> >>> > the connection is pulled and held for quite a while. Imagine an
> >>> app
> >>> > has 10 database controls that point to the same database. If they
> >>> > are used in the same Pageflow, it eats up 10 connections to the
> >>> > database. That's a lot of overhead for one Pageflow to inflict
> >>> on a
> >>> > DataSource.
> >>> >
> >>> > Is there a reason not to move getConnection() at the beginning of
> >>> > invoke() and then close it out at the end of invoke()? This would
> >>> > be nice for JDBC controls with ConnectionDataSource
> >>> annotations. If
> >>> > there is some reason to hold on the connection like the current
> >>> > implementation, could we set up an annotation to allow this
> >>> > connection-conserving behavior?
> >>> >
> >>> > - Drew
> >>> >
> >>> > On Dec 1, 2006, at 10:07 AM, Eddie O'Neil wrote:
> >>> >
> >>> >> Drew--
> >>> >>
> >>> >> Definitely -- this would be a great feature that would help
> >>> with the
> >>> >> situations you described. Today, it's not possible to manage
> >>> the
> >>> >> Connection directly which makes it impossible to share it
> >>> between two
> >>> >> JdbcControl instances. It might be possible to just add a method
> >>> >> like:
> >>> >>
> >>> >> public void setConnection(Connection conn)
> >>> >>
> >>> >> that if called will short circuit the Connection creation code
> >>> in the
> >>> >> JdbcControl.
> >>> >>
> >>> >> This might be easy to add to the Control -- feel free to give it
> >>> >> a shot. :)
> >>> >>
> >>> >> Eddie
> >>> >>
> >>> >>
> >>> >> On 11/30/06, Drew Varner <dr...@redops.org> wrote:
> >>> >>> I was playing around with an idea for the JDBC system
> >>> control. The
> >>> >>> ability to pass it a Connection object other than one
> >>> declared in
> >>> >>> it's annotations.
> >>> >>>
> >>> >>> Here are a couple of use cases:
> >>> >>>
> >>> >>> 1) Integration with other persistence mechanisms. Let's say I
> >>> have a
> >>> >>> DAO that I can pass a Connection to. I can place the DAO and and
> >>> >>> JDBC
> >>> >>> Control operations in the same transaction.
> >>> >>>
> >>> >>> Connection conn = JDBCUtils.getConnection();
> >>> >>> conn.setAutoCommit(false);
> >>> >>> myJdbcControl.setExternalConnection(conn);
> >>> >>> myJdbcControl.depositMoney(1000)
> >>> >>> AccountAuditDAO.logDepositTransaction(1000,conn);
> >>> >>> conn.commit();
> >>> >>> conn.close();
> >>> >>>
> >>> >>> 2) Use it to put multiple JDBC Controls in the same db
> >>> transaction
> >>> >>> (same as above, replace the DAO with another JDBC control)
> >>> >>>
> >>> >>> 3) Make unit testing easier. You can pass a connection to a test
> >>> >>> database into the control during testing and don't have to
> >>> mess with
> >>> >>> the JNDI setup for out-of-container testing.
> >>> >>>
> >>> >>> 4) Support for non-standard Connection sources. If you are
> >>> using a
> >>> >>> JDBC Control in a Struts app, you can grab the DataSource via
> >>> the
> >>> >>> Struts DataSource mechanism.
> >>> >>>
> >>> >>> Setting an external connection would be manipulated by
> >>> methods, not
> >>> >>> annotations.
> >>> >>>
> >>> >>> public void setExternalConnection(Connection conn);
> >>> >>> public Connection getExternalConnection();
> >>> >>>
> >>> >>> The control implementation would never close the external
> >>> >>> connection.
> >>> >>>
> >>> >>> Would it be worth adding?
> >>> >>>
> >>> >>> - Drew
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >
> >>>
> >>>
> >
>
>
Re: Idea for JDBC system control
Posted by Drew Varner <dr...@redops.org>.
Ignore the "must setup JNDI if you don't need it" at the end, you
must setup JNDI or explicitly null it out via a bean method.
- Drew
On Dec 13, 2006, at 1:28 AM, Drew Varner wrote:
> Hey Chad,
>
> I was not sure where those events fired.
>
> My concern is picking up multiple connections where one is needed.
> In an example where one database control is used the results are no
> different than using any other persistence mechanism. However, if
> we have many controls, it becomes problematic. If I use controls as
> my basic building block for data abstraction, I could have one per
> domain object. It's not beyond reason to expect 10 connections to
> be used in a method. This is definitely the case where we wrap
> multiple JDBC Controls in a custom control. Asking the architect to
> accept a database connection pool five times or more than that of
> another persistence solution seems steep. When load surges, you may
> need one file descriptor per request to service the HTTP request
> but ten to connect to the database. Manipulating 10 domain objects
> in a call may seem extreme,
>
> We could work around this problem by creating one control per
> database. This control would quickly become a maintenance monster.
>
> There are other applications for controls outside of short-lived
> HTTP requests. BEA's WebLogic Integration product uses Controls
> extensively in JPDs. I can't speak to the details of a JpdConext
> but they don't sound like short lifespans. Assuming a rich data
> model, the impact of a control keeping a tight grip on a Connection
> could be high.
>
> I think we know whether or not it is safe to close the connection
> after use. The ResultSetMapper's canCloseResultSet() should be
> enough information. If you can close the ResultSet and the
> Connection is in auto-commit mode, you should be able to close it
> safely (according to the unit tests).
>
> While I appreciate that part of the controls mantra is
> simplification when it comes to managing resources, DataSources
> probably do a better job than a control of managing Connections
> (running queries to identify stale connections, etc.). But, if we
> limit controls to Pageflows or other web applications, these
> features aren't a big deal.
>
> Having the database connection instantiated lazily makes testing
> easier. Look at Chris' example of testing a Control with a JNDI
> DataSource:
>
> http://dev2dev.bea.com/blog/hogue/archive/2006/07/
> outofcontainer_1.html
>
> You have to reference the bean, not the interface. You have to
> setup a JNDI Context if you use it in testing or not. If you lazily
> instantiate, these hassles go away.
>
> - Drew
>
> On Dec 12, 2006, at 11:13 PM, Chad Schoettger wrote:
>
>> Hi Drew,
>>
>> I haven't had a chance to look at your patch yet (hopefully I'll have
>> some time tomorrow) but I think the best solution may be to leave the
>> aquisition/release of the connection in the onAcquire/onRelease
>> methods.
>>
>> When using a JDBC control from a page flow these events provide a
>> reasonably nice way of managing the connections. The connection is
>> basically scoped for the page flow method the control is being
>> invoked
>> from:
>>
>> MyPageFlow.java
>>
>> @Control private JDBCControl _myJdbcControl;
>>
>> myAction() {
>> _myJdbcControl.getRow() /* causes onAcquire() to be
>> invoked */
>> _myJdbcControl.getAnotherRow() /* does NOT cause onAquire() to
>> be invoked */
>> ......
>> ......
>> return .... /* causes onRelease() to be invoked() */
>> }
>>
>> I really like the idea Eddie mentioned about having a set method for
>> the connection -- if the user sets the connection, the jdbc control
>> closes an open connection if it has one and uses the one provided by
>> the user and basically noops in the onAcquire/onRelease methods. If
>> the user sets the connection to null, the onAcquire/onRelease default
>> behavior resumes.
>>
>> I do apologize if you were already aware of when onAcquire/onRelease
>> were invoked, it seems to be an area which causes a fair amount of
>> confusion.
>>
>> - Chad
>>
>>
>>
>>
>> On 12/11/06, Drew Varner <dr...@redops.org> wrote:
>>> Ok, I now see why the connection cannot be closed immediately after
>>> invoke: ResultSet results and possibly connected RowSets.
>>>
>>> Would it be possible to flag SQL methods to indicate that they don't
>>> depend on an open connection? This option would not break existing
>>> implementations.
>>>
>>> Another option would be to check the return type of the method. If
>>> the return type is ResultSet or RowSet, we don't close it. I don't
>>> like this solution as much. If a custom mapper does something weird
>>> like wrapping a RowSet, we'll never know. This solution seems like
>>> more of a hack.
>>>
>>> - Drew
>>>
>>>
>>> On Dec 11, 2006, at 10:57 PM, Drew Varner wrote:
>>>
>>> > I attached the patch to BEEHIVE-1019 (http://issues.apache.org/
>>> jira/
>>> > browse/BEEHIVE-1019).
>>> >
>>> > The patch I attached moves getConnection() from onAcquire() to
>>> > invoke().
>>> >
>>> > In the current version (grab connection when control is acquired)
>>> > or the new version (if my patch is accepted) of the JDBC Control,
>>> > the connection is pulled and held for quite a while. Imagine an
>>> app
>>> > has 10 database controls that point to the same database. If they
>>> > are used in the same Pageflow, it eats up 10 connections to the
>>> > database. That's a lot of overhead for one Pageflow to inflict
>>> on a
>>> > DataSource.
>>> >
>>> > Is there a reason not to move getConnection() at the beginning of
>>> > invoke() and then close it out at the end of invoke()? This would
>>> > be nice for JDBC controls with ConnectionDataSource
>>> annotations. If
>>> > there is some reason to hold on the connection like the current
>>> > implementation, could we set up an annotation to allow this
>>> > connection-conserving behavior?
>>> >
>>> > - Drew
>>> >
>>> > On Dec 1, 2006, at 10:07 AM, Eddie O'Neil wrote:
>>> >
>>> >> Drew--
>>> >>
>>> >> Definitely -- this would be a great feature that would help
>>> with the
>>> >> situations you described. Today, it's not possible to manage
>>> the
>>> >> Connection directly which makes it impossible to share it
>>> between two
>>> >> JdbcControl instances. It might be possible to just add a method
>>> >> like:
>>> >>
>>> >> public void setConnection(Connection conn)
>>> >>
>>> >> that if called will short circuit the Connection creation code
>>> in the
>>> >> JdbcControl.
>>> >>
>>> >> This might be easy to add to the Control -- feel free to give it
>>> >> a shot. :)
>>> >>
>>> >> Eddie
>>> >>
>>> >>
>>> >> On 11/30/06, Drew Varner <dr...@redops.org> wrote:
>>> >>> I was playing around with an idea for the JDBC system
>>> control. The
>>> >>> ability to pass it a Connection object other than one
>>> declared in
>>> >>> it's annotations.
>>> >>>
>>> >>> Here are a couple of use cases:
>>> >>>
>>> >>> 1) Integration with other persistence mechanisms. Let's say I
>>> have a
>>> >>> DAO that I can pass a Connection to. I can place the DAO and and
>>> >>> JDBC
>>> >>> Control operations in the same transaction.
>>> >>>
>>> >>> Connection conn = JDBCUtils.getConnection();
>>> >>> conn.setAutoCommit(false);
>>> >>> myJdbcControl.setExternalConnection(conn);
>>> >>> myJdbcControl.depositMoney(1000)
>>> >>> AccountAuditDAO.logDepositTransaction(1000,conn);
>>> >>> conn.commit();
>>> >>> conn.close();
>>> >>>
>>> >>> 2) Use it to put multiple JDBC Controls in the same db
>>> transaction
>>> >>> (same as above, replace the DAO with another JDBC control)
>>> >>>
>>> >>> 3) Make unit testing easier. You can pass a connection to a test
>>> >>> database into the control during testing and don't have to
>>> mess with
>>> >>> the JNDI setup for out-of-container testing.
>>> >>>
>>> >>> 4) Support for non-standard Connection sources. If you are
>>> using a
>>> >>> JDBC Control in a Struts app, you can grab the DataSource via
>>> the
>>> >>> Struts DataSource mechanism.
>>> >>>
>>> >>> Setting an external connection would be manipulated by
>>> methods, not
>>> >>> annotations.
>>> >>>
>>> >>> public void setExternalConnection(Connection conn);
>>> >>> public Connection getExternalConnection();
>>> >>>
>>> >>> The control implementation would never close the external
>>> >>> connection.
>>> >>>
>>> >>> Would it be worth adding?
>>> >>>
>>> >>> - Drew
>>> >>>
>>> >>>
>>> >>>
>>> >
>>>
>>>
>
Re: Idea for JDBC system control
Posted by Drew Varner <dr...@redops.org>.
Hey Chad,
I was not sure where those events fired.
My concern is picking up multiple connections where one is needed. In
an example where one database control is used the results are no
different than using any other persistence mechanism. However, if we
have many controls, it becomes problematic. If I use controls as my
basic building block for data abstraction, I could have one per
domain object. It's not beyond reason to expect 10 connections to be
used in a method. This is definitely the case where we wrap multiple
JDBC Controls in a custom control. Asking the architect to accept a
database connection pool five times or more than that of another
persistence solution seems steep. When load surges, you may need one
file descriptor per request to service the HTTP request but ten to
connect to the database. Manipulating 10 domain objects in a call may
seem extreme,
We could work around this problem by creating one control per
database. This control would quickly become a maintenance monster.
There are other applications for controls outside of short-lived HTTP
requests. BEA's WebLogic Integration product uses Controls
extensively in JPDs. I can't speak to the details of a JpdConext but
they don't sound like short lifespans. Assuming a rich data model,
the impact of a control keeping a tight grip on a Connection could be
high.
I think we know whether or not it is safe to close the connection
after use. The ResultSetMapper's canCloseResultSet() should be enough
information. If you can close the ResultSet and the Connection is in
auto-commit mode, you should be able to close it safely (according to
the unit tests).
While I appreciate that part of the controls mantra is simplification
when it comes to managing resources, DataSources probably do a better
job than a control of managing Connections (running queries to
identify stale connections, etc.). But, if we limit controls to
Pageflows or other web applications, these features aren't a big deal.
Having the database connection instantiated lazily makes testing
easier. Look at Chris' example of testing a Control with a JNDI
DataSource:
http://dev2dev.bea.com/blog/hogue/archive/2006/07/outofcontainer_1.html
You have to reference the bean, not the interface. You have to setup
a JNDI Context if you use it in testing or not. If you lazily
instantiate, these hassles go away.
- Drew
On Dec 12, 2006, at 11:13 PM, Chad Schoettger wrote:
> Hi Drew,
>
> I haven't had a chance to look at your patch yet (hopefully I'll have
> some time tomorrow) but I think the best solution may be to leave the
> aquisition/release of the connection in the onAcquire/onRelease
> methods.
>
> When using a JDBC control from a page flow these events provide a
> reasonably nice way of managing the connections. The connection is
> basically scoped for the page flow method the control is being invoked
> from:
>
> MyPageFlow.java
>
> @Control private JDBCControl _myJdbcControl;
>
> myAction() {
> _myJdbcControl.getRow() /* causes onAcquire() to be invoked */
> _myJdbcControl.getAnotherRow() /* does NOT cause onAquire() to
> be invoked */
> ......
> ......
> return .... /* causes onRelease() to be invoked() */
> }
>
> I really like the idea Eddie mentioned about having a set method for
> the connection -- if the user sets the connection, the jdbc control
> closes an open connection if it has one and uses the one provided by
> the user and basically noops in the onAcquire/onRelease methods. If
> the user sets the connection to null, the onAcquire/onRelease default
> behavior resumes.
>
> I do apologize if you were already aware of when onAcquire/onRelease
> were invoked, it seems to be an area which causes a fair amount of
> confusion.
>
> - Chad
>
>
>
>
> On 12/11/06, Drew Varner <dr...@redops.org> wrote:
>> Ok, I now see why the connection cannot be closed immediately after
>> invoke: ResultSet results and possibly connected RowSets.
>>
>> Would it be possible to flag SQL methods to indicate that they don't
>> depend on an open connection? This option would not break existing
>> implementations.
>>
>> Another option would be to check the return type of the method. If
>> the return type is ResultSet or RowSet, we don't close it. I don't
>> like this solution as much. If a custom mapper does something weird
>> like wrapping a RowSet, we'll never know. This solution seems like
>> more of a hack.
>>
>> - Drew
>>
>>
>> On Dec 11, 2006, at 10:57 PM, Drew Varner wrote:
>>
>> > I attached the patch to BEEHIVE-1019 (http://issues.apache.org/
>> jira/
>> > browse/BEEHIVE-1019).
>> >
>> > The patch I attached moves getConnection() from onAcquire() to
>> > invoke().
>> >
>> > In the current version (grab connection when control is acquired)
>> > or the new version (if my patch is accepted) of the JDBC Control,
>> > the connection is pulled and held for quite a while. Imagine an app
>> > has 10 database controls that point to the same database. If they
>> > are used in the same Pageflow, it eats up 10 connections to the
>> > database. That's a lot of overhead for one Pageflow to inflict on a
>> > DataSource.
>> >
>> > Is there a reason not to move getConnection() at the beginning of
>> > invoke() and then close it out at the end of invoke()? This would
>> > be nice for JDBC controls with ConnectionDataSource annotations. If
>> > there is some reason to hold on the connection like the current
>> > implementation, could we set up an annotation to allow this
>> > connection-conserving behavior?
>> >
>> > - Drew
>> >
>> > On Dec 1, 2006, at 10:07 AM, Eddie O'Neil wrote:
>> >
>> >> Drew--
>> >>
>> >> Definitely -- this would be a great feature that would help
>> with the
>> >> situations you described. Today, it's not possible to manage the
>> >> Connection directly which makes it impossible to share it
>> between two
>> >> JdbcControl instances. It might be possible to just add a method
>> >> like:
>> >>
>> >> public void setConnection(Connection conn)
>> >>
>> >> that if called will short circuit the Connection creation code
>> in the
>> >> JdbcControl.
>> >>
>> >> This might be easy to add to the Control -- feel free to give it
>> >> a shot. :)
>> >>
>> >> Eddie
>> >>
>> >>
>> >> On 11/30/06, Drew Varner <dr...@redops.org> wrote:
>> >>> I was playing around with an idea for the JDBC system control.
>> The
>> >>> ability to pass it a Connection object other than one declared in
>> >>> it's annotations.
>> >>>
>> >>> Here are a couple of use cases:
>> >>>
>> >>> 1) Integration with other persistence mechanisms. Let's say I
>> have a
>> >>> DAO that I can pass a Connection to. I can place the DAO and and
>> >>> JDBC
>> >>> Control operations in the same transaction.
>> >>>
>> >>> Connection conn = JDBCUtils.getConnection();
>> >>> conn.setAutoCommit(false);
>> >>> myJdbcControl.setExternalConnection(conn);
>> >>> myJdbcControl.depositMoney(1000)
>> >>> AccountAuditDAO.logDepositTransaction(1000,conn);
>> >>> conn.commit();
>> >>> conn.close();
>> >>>
>> >>> 2) Use it to put multiple JDBC Controls in the same db
>> transaction
>> >>> (same as above, replace the DAO with another JDBC control)
>> >>>
>> >>> 3) Make unit testing easier. You can pass a connection to a test
>> >>> database into the control during testing and don't have to
>> mess with
>> >>> the JNDI setup for out-of-container testing.
>> >>>
>> >>> 4) Support for non-standard Connection sources. If you are
>> using a
>> >>> JDBC Control in a Struts app, you can grab the DataSource via the
>> >>> Struts DataSource mechanism.
>> >>>
>> >>> Setting an external connection would be manipulated by
>> methods, not
>> >>> annotations.
>> >>>
>> >>> public void setExternalConnection(Connection conn);
>> >>> public Connection getExternalConnection();
>> >>>
>> >>> The control implementation would never close the external
>> >>> connection.
>> >>>
>> >>> Would it be worth adding?
>> >>>
>> >>> - Drew
>> >>>
>> >>>
>> >>>
>> >
>>
>>
Re: Idea for JDBC system control
Posted by Chad Schoettger <ch...@gmail.com>.
Hi Drew,
I haven't had a chance to look at your patch yet (hopefully I'll have
some time tomorrow) but I think the best solution may be to leave the
aquisition/release of the connection in the onAcquire/onRelease
methods.
When using a JDBC control from a page flow these events provide a
reasonably nice way of managing the connections. The connection is
basically scoped for the page flow method the control is being invoked
from:
MyPageFlow.java
@Control private JDBCControl _myJdbcControl;
myAction() {
_myJdbcControl.getRow() /* causes onAcquire() to be invoked */
_myJdbcControl.getAnotherRow() /* does NOT cause onAquire() to
be invoked */
......
......
return .... /* causes onRelease() to be invoked() */
}
I really like the idea Eddie mentioned about having a set method for
the connection -- if the user sets the connection, the jdbc control
closes an open connection if it has one and uses the one provided by
the user and basically noops in the onAcquire/onRelease methods. If
the user sets the connection to null, the onAcquire/onRelease default
behavior resumes.
I do apologize if you were already aware of when onAcquire/onRelease
were invoked, it seems to be an area which causes a fair amount of
confusion.
- Chad
On 12/11/06, Drew Varner <dr...@redops.org> wrote:
> Ok, I now see why the connection cannot be closed immediately after
> invoke: ResultSet results and possibly connected RowSets.
>
> Would it be possible to flag SQL methods to indicate that they don't
> depend on an open connection? This option would not break existing
> implementations.
>
> Another option would be to check the return type of the method. If
> the return type is ResultSet or RowSet, we don't close it. I don't
> like this solution as much. If a custom mapper does something weird
> like wrapping a RowSet, we'll never know. This solution seems like
> more of a hack.
>
> - Drew
>
>
> On Dec 11, 2006, at 10:57 PM, Drew Varner wrote:
>
> > I attached the patch to BEEHIVE-1019 (http://issues.apache.org/jira/
> > browse/BEEHIVE-1019).
> >
> > The patch I attached moves getConnection() from onAcquire() to
> > invoke().
> >
> > In the current version (grab connection when control is acquired)
> > or the new version (if my patch is accepted) of the JDBC Control,
> > the connection is pulled and held for quite a while. Imagine an app
> > has 10 database controls that point to the same database. If they
> > are used in the same Pageflow, it eats up 10 connections to the
> > database. That's a lot of overhead for one Pageflow to inflict on a
> > DataSource.
> >
> > Is there a reason not to move getConnection() at the beginning of
> > invoke() and then close it out at the end of invoke()? This would
> > be nice for JDBC controls with ConnectionDataSource annotations. If
> > there is some reason to hold on the connection like the current
> > implementation, could we set up an annotation to allow this
> > connection-conserving behavior?
> >
> > - Drew
> >
> > On Dec 1, 2006, at 10:07 AM, Eddie O'Neil wrote:
> >
> >> Drew--
> >>
> >> Definitely -- this would be a great feature that would help with the
> >> situations you described. Today, it's not possible to manage the
> >> Connection directly which makes it impossible to share it between two
> >> JdbcControl instances. It might be possible to just add a method
> >> like:
> >>
> >> public void setConnection(Connection conn)
> >>
> >> that if called will short circuit the Connection creation code in the
> >> JdbcControl.
> >>
> >> This might be easy to add to the Control -- feel free to give it
> >> a shot. :)
> >>
> >> Eddie
> >>
> >>
> >> On 11/30/06, Drew Varner <dr...@redops.org> wrote:
> >>> I was playing around with an idea for the JDBC system control. The
> >>> ability to pass it a Connection object other than one declared in
> >>> it's annotations.
> >>>
> >>> Here are a couple of use cases:
> >>>
> >>> 1) Integration with other persistence mechanisms. Let's say I have a
> >>> DAO that I can pass a Connection to. I can place the DAO and and
> >>> JDBC
> >>> Control operations in the same transaction.
> >>>
> >>> Connection conn = JDBCUtils.getConnection();
> >>> conn.setAutoCommit(false);
> >>> myJdbcControl.setExternalConnection(conn);
> >>> myJdbcControl.depositMoney(1000)
> >>> AccountAuditDAO.logDepositTransaction(1000,conn);
> >>> conn.commit();
> >>> conn.close();
> >>>
> >>> 2) Use it to put multiple JDBC Controls in the same db transaction
> >>> (same as above, replace the DAO with another JDBC control)
> >>>
> >>> 3) Make unit testing easier. You can pass a connection to a test
> >>> database into the control during testing and don't have to mess with
> >>> the JNDI setup for out-of-container testing.
> >>>
> >>> 4) Support for non-standard Connection sources. If you are using a
> >>> JDBC Control in a Struts app, you can grab the DataSource via the
> >>> Struts DataSource mechanism.
> >>>
> >>> Setting an external connection would be manipulated by methods, not
> >>> annotations.
> >>>
> >>> public void setExternalConnection(Connection conn);
> >>> public Connection getExternalConnection();
> >>>
> >>> The control implementation would never close the external
> >>> connection.
> >>>
> >>> Would it be worth adding?
> >>>
> >>> - Drew
> >>>
> >>>
> >>>
> >
>
>
Re: Idea for JDBC system control
Posted by Drew Varner <dr...@redops.org>.
Ok, I now see why the connection cannot be closed immediately after
invoke: ResultSet results and possibly connected RowSets.
Would it be possible to flag SQL methods to indicate that they don't
depend on an open connection? This option would not break existing
implementations.
Another option would be to check the return type of the method. If
the return type is ResultSet or RowSet, we don't close it. I don't
like this solution as much. If a custom mapper does something weird
like wrapping a RowSet, we'll never know. This solution seems like
more of a hack.
- Drew
On Dec 11, 2006, at 10:57 PM, Drew Varner wrote:
> I attached the patch to BEEHIVE-1019 (http://issues.apache.org/jira/
> browse/BEEHIVE-1019).
>
> The patch I attached moves getConnection() from onAcquire() to
> invoke().
>
> In the current version (grab connection when control is acquired)
> or the new version (if my patch is accepted) of the JDBC Control,
> the connection is pulled and held for quite a while. Imagine an app
> has 10 database controls that point to the same database. If they
> are used in the same Pageflow, it eats up 10 connections to the
> database. That's a lot of overhead for one Pageflow to inflict on a
> DataSource.
>
> Is there a reason not to move getConnection() at the beginning of
> invoke() and then close it out at the end of invoke()? This would
> be nice for JDBC controls with ConnectionDataSource annotations. If
> there is some reason to hold on the connection like the current
> implementation, could we set up an annotation to allow this
> connection-conserving behavior?
>
> - Drew
>
> On Dec 1, 2006, at 10:07 AM, Eddie O'Neil wrote:
>
>> Drew--
>>
>> Definitely -- this would be a great feature that would help with the
>> situations you described. Today, it's not possible to manage the
>> Connection directly which makes it impossible to share it between two
>> JdbcControl instances. It might be possible to just add a method
>> like:
>>
>> public void setConnection(Connection conn)
>>
>> that if called will short circuit the Connection creation code in the
>> JdbcControl.
>>
>> This might be easy to add to the Control -- feel free to give it
>> a shot. :)
>>
>> Eddie
>>
>>
>> On 11/30/06, Drew Varner <dr...@redops.org> wrote:
>>> I was playing around with an idea for the JDBC system control. The
>>> ability to pass it a Connection object other than one declared in
>>> it's annotations.
>>>
>>> Here are a couple of use cases:
>>>
>>> 1) Integration with other persistence mechanisms. Let's say I have a
>>> DAO that I can pass a Connection to. I can place the DAO and and
>>> JDBC
>>> Control operations in the same transaction.
>>>
>>> Connection conn = JDBCUtils.getConnection();
>>> conn.setAutoCommit(false);
>>> myJdbcControl.setExternalConnection(conn);
>>> myJdbcControl.depositMoney(1000)
>>> AccountAuditDAO.logDepositTransaction(1000,conn);
>>> conn.commit();
>>> conn.close();
>>>
>>> 2) Use it to put multiple JDBC Controls in the same db transaction
>>> (same as above, replace the DAO with another JDBC control)
>>>
>>> 3) Make unit testing easier. You can pass a connection to a test
>>> database into the control during testing and don't have to mess with
>>> the JNDI setup for out-of-container testing.
>>>
>>> 4) Support for non-standard Connection sources. If you are using a
>>> JDBC Control in a Struts app, you can grab the DataSource via the
>>> Struts DataSource mechanism.
>>>
>>> Setting an external connection would be manipulated by methods, not
>>> annotations.
>>>
>>> public void setExternalConnection(Connection conn);
>>> public Connection getExternalConnection();
>>>
>>> The control implementation would never close the external
>>> connection.
>>>
>>> Would it be worth adding?
>>>
>>> - Drew
>>>
>>>
>>>
>
Re: Idea for JDBC system control
Posted by Drew Varner <dr...@redops.org>.
I attached the patch to BEEHIVE-1019 (http://issues.apache.org/jira/
browse/BEEHIVE-1019).
The patch I attached moves getConnection() from onAcquire() to invoke().
In the current version (grab connection when control is acquired) or
the new version (if my patch is accepted) of the JDBC Control, the
connection is pulled and held for quite a while. Imagine an app has
10 database controls that point to the same database. If they are
used in the same Pageflow, it eats up 10 connections to the database.
That's a lot of overhead for one Pageflow to inflict on a DataSource.
Is there a reason not to move getConnection() at the beginning of
invoke() and then close it out at the end of invoke()? This would be
nice for JDBC controls with ConnectionDataSource annotations. If
there is some reason to hold on the connection like the current
implementation, could we set up an annotation to allow this
connection-conserving behavior?
- Drew
On Dec 1, 2006, at 10:07 AM, Eddie O'Neil wrote:
> Drew--
>
> Definitely -- this would be a great feature that would help with the
> situations you described. Today, it's not possible to manage the
> Connection directly which makes it impossible to share it between two
> JdbcControl instances. It might be possible to just add a method
> like:
>
> public void setConnection(Connection conn)
>
> that if called will short circuit the Connection creation code in the
> JdbcControl.
>
> This might be easy to add to the Control -- feel free to give it a
> shot. :)
>
> Eddie
>
>
> On 11/30/06, Drew Varner <dr...@redops.org> wrote:
>> I was playing around with an idea for the JDBC system control. The
>> ability to pass it a Connection object other than one declared in
>> it's annotations.
>>
>> Here are a couple of use cases:
>>
>> 1) Integration with other persistence mechanisms. Let's say I have a
>> DAO that I can pass a Connection to. I can place the DAO and and JDBC
>> Control operations in the same transaction.
>>
>> Connection conn = JDBCUtils.getConnection();
>> conn.setAutoCommit(false);
>> myJdbcControl.setExternalConnection(conn);
>> myJdbcControl.depositMoney(1000)
>> AccountAuditDAO.logDepositTransaction(1000,conn);
>> conn.commit();
>> conn.close();
>>
>> 2) Use it to put multiple JDBC Controls in the same db transaction
>> (same as above, replace the DAO with another JDBC control)
>>
>> 3) Make unit testing easier. You can pass a connection to a test
>> database into the control during testing and don't have to mess with
>> the JNDI setup for out-of-container testing.
>>
>> 4) Support for non-standard Connection sources. If you are using a
>> JDBC Control in a Struts app, you can grab the DataSource via the
>> Struts DataSource mechanism.
>>
>> Setting an external connection would be manipulated by methods, not
>> annotations.
>>
>> public void setExternalConnection(Connection conn);
>> public Connection getExternalConnection();
>>
>> The control implementation would never close the external connection.
>>
>> Would it be worth adding?
>>
>> - Drew
>>
>>
>>
Re: Idea for JDBC system control
Posted by Eddie O'Neil <ek...@gmail.com>.
Drew--
Definitely -- this would be a great feature that would help with the
situations you described. Today, it's not possible to manage the
Connection directly which makes it impossible to share it between two
JdbcControl instances. It might be possible to just add a method
like:
public void setConnection(Connection conn)
that if called will short circuit the Connection creation code in the
JdbcControl.
This might be easy to add to the Control -- feel free to give it a shot. :)
Eddie
On 11/30/06, Drew Varner <dr...@redops.org> wrote:
> I was playing around with an idea for the JDBC system control. The
> ability to pass it a Connection object other than one declared in
> it's annotations.
>
> Here are a couple of use cases:
>
> 1) Integration with other persistence mechanisms. Let's say I have a
> DAO that I can pass a Connection to. I can place the DAO and and JDBC
> Control operations in the same transaction.
>
> Connection conn = JDBCUtils.getConnection();
> conn.setAutoCommit(false);
> myJdbcControl.setExternalConnection(conn);
> myJdbcControl.depositMoney(1000)
> AccountAuditDAO.logDepositTransaction(1000,conn);
> conn.commit();
> conn.close();
>
> 2) Use it to put multiple JDBC Controls in the same db transaction
> (same as above, replace the DAO with another JDBC control)
>
> 3) Make unit testing easier. You can pass a connection to a test
> database into the control during testing and don't have to mess with
> the JNDI setup for out-of-container testing.
>
> 4) Support for non-standard Connection sources. If you are using a
> JDBC Control in a Struts app, you can grab the DataSource via the
> Struts DataSource mechanism.
>
> Setting an external connection would be manipulated by methods, not
> annotations.
>
> public void setExternalConnection(Connection conn);
> public Connection getExternalConnection();
>
> The control implementation would never close the external connection.
>
> Would it be worth adding?
>
> - Drew
>
>
>