Implementing Statement.cancel()

[Oy...@Sun.COM]

I've been looking at implementing Statement.cancel(), and later, 
Statement.setQueryTimeout().

My proposal is as follows:
* Add these two methods to org.apache.derby.iapi.sql.conn.StatementContext:
   public boolean isCancelled();
   public void cancel();
* Implement these in 
org.apache.derby.impl.sql.conn.GenericStatementContext, simply operating 
on a private boolean flag (with proper synchronization). Of course, 
clear it in the clearInUse() method.
* From the getNextRowCore() method of various "bottom-level" (referring 
to the ResultSet hierarchy) ResultSet implementations (such as 
[Bulk]TableScanResultSet), get the executing statement's 
StatementContext object (see below) and call its isCancelled() method. 
If it returns true, throw 
StandardException.newException(SQLState.LANG_STATEMENT_CLOSED).
* From EmbedStatement.cancel(), get the statement's StatementContext 
object (see below) and call its cancel() method. (First check the 
statement's active flag.)

I have tested this and it seems to work fine according to the semantics 
we discussed in earlier mails. However, I am unsure of the correct way 
to "get the executing statement's StatementContext object".

 From the ResultSet side, the BasicNoPutResultSet class contains a 
declaration

   // set on demand during execution
   private StatementContext statementContext;

but this reference has the value null where I need it. There exists a 
method attachStatementContext(), but this does no assignment unless the 
isTopResultSet attribute is true - which it rarely is in my case.

 From the EmbedStatement class, there is no direct reference to a 
StatementContext object.

What I have been testing so far is using the 
LanguageConnectionContext.getStatementContext() method through the lcc 
reference available to both EmbedStatement and BasicNoPutResultSet. 
However, it seems odd to me to go through a "connection context" object 
to get the statement context. I'm not sure this is the right way to do 
it. What if, for instance, there are multiple open statements on the 
same connection? What statement's context object is returned?

-- 
Øyvind Bakksjø
Sun Microsystems, Web Services, Database Technology Group
Haakon VII gt. 7b, N-7485 Trondheim, Norway
Tel: x43419 / +47 73842119, Fax: +47 73842101

Re: Implementing Statement.cancel()

[Oy...@Sun.COM]

Daniel John Debrunner wrote:
> Oyvind.Bakksjo@Sun.COM wrote:
> 
> 
>>I've been looking at implementing Statement.cancel(), and later,
>>Statement.setQueryTimeout().
> 
> 
> I'll look into how I think you should track down the StatementContext.
> Dan.

Hi Dan,

have you had time to look into this?

-- 
Øyvind Bakksjø
Sun Microsystems, Web Services, Database Technology Group
Haakon VII gt. 7b, N-7485 Trondheim, Norway
Tel: x43419 / +47 73842119, Fax: +47 73842101

Re: Implementing Statement.cancel()

[Oy...@Sun.COM]

Daniel John Debrunner wrote:
> Oyvind.Bakksjo@sun.com wrote:
> 
> 
>>Daniel John Debrunner wrote:
>>
>>>I'll look into how I think you should track down the StatementContext.
>>>Dan.
>>
>>
>>Hi Dan,
>>
>>have you had any time to look into this?
> 
> Not a lot :-(

So I assumed, so I started digging deeper into it myself. I wrongfully 
assumed that a statement execution had a context object for its 
lifetime. Once I found out that the StatementContext object is assigned 
only when the statement is actively executing (setInUse/clearInUse is 
called, for instance, for each fetch from the ResultSet), I realized 
that there might not *be* an assigned StatementContext object when a 
thread calls cancel(). Therefore, I'll have to come up with a different 
solution for the cancel method.

On the other hand, I now have a working implementation of 
setQueryTimeout (which is actually what the DERBY-31 JIRA issue is 
requesting). cancel() is not a prerequisite for setQueryTimeout(). 
Statement execution propagates the timeout value down to the point where 
a StatementContext object is assigned (with setInUse), and a TimerTask 
(which, on timeout, sets a cancellation flag in the StatementContext 
object) is scheduled on a system-wide Timer. Hence, there'll be one more 
thread in the JVM per loaded Derby system (not per database). On 
clearInUse(), the TimerTask is cancelled.

I'm in the process of polishing a patch and writing functional tests for 
setQueryTimeout.

> I've been thinking about it and there are a couple of issues:
> 
> 1) StatementContexts are created dynamically for the current running
> statement
> 
> 2) Derby does not keep references to application JDBC objects below the
> JDBC layer. This is to ensure when any JDBC object (e.g. Statement)
> moves out of scope for the application, it will be garbage collected and
> lead to its closure.
> 
> I wonder if from a StatementContext you can determine the activation
> because that would provide the link to the Statement.

Maybe that will be needed for implementing cancel(). I'll look into it 
when I've finished the current patch for setQueryTimeout().

-- 
Øyvind Bakksjø
Sun Microsystems, Web Services, Database Technology Group
Haakon VII gt. 7b, N-7485 Trondheim, Norway
Tel: x43419 / +47 73842119, Fax: +47 73842101

Re: Implementing Statement.cancel()

[David Van Couvering <Da...@Sun.COM>]

I have had similar difficulties finding navigation paths from one object 
to another within the domain of execution contexts.

I have been toying with the idea of putting together one or two UML 
class diagrams (using an open-source tool, of course :)) to try and show 
the relationships between all these numerous classes and interfaces. 
Every time I get in there I have to re-learn how the pieces fit 
together, and often my internal buffer overflows and I get lost :)

It would also be great to document some of the design 
principles/approaches to this layer of the code, as it's not at all 
clear through reading the code.  I don't have this knowledge, but if 
someone who does can type something up, I can try and put it into a form 
that can be posted as a paper to the web site.

David

Daniel John Debrunner wrote:

> Oyvind.Bakksjo@sun.com wrote:
> 
> 
>>Daniel John Debrunner wrote:
>>
>>
>>>Oyvind.Bakksjo@Sun.COM wrote:
>>>
>>>
>>>
>>>>I've been looking at implementing Statement.cancel(), and later,
>>>>Statement.setQueryTimeout().
>>>
>>>
>>>
>>>So, quick summary, that method is not suitable for finding the correct
>>>StatementContext from EmbedStatement.cancel().
>>>
>>>I'll look into how I think you should track down the StatementContext.
>>>Dan.
>>
>>
>>Hi Dan,
>>
>>have you had any time to look into this?
> 
> Not a lot :-(
> 
> I've been thinking about it and there are a couple of issues:
> 
> 1) StatementContexts are created dynamically for the current running
> statement
> 
> 2) Derby does not keep references to application JDBC objects below the
> JDBC layer. This is to ensure when any JDBC object (e.g. Statement)
> moves out of scope for the application, it will be garbage collected and
> lead to its closure.
> 
> I wonder if from a StatementContext you can determine the activation
> because that would provide the link to the Statement.
> 
> 
> Dan.
> 
> 
> 

Re: Implementing Statement.cancel()

[Daniel John Debrunner <dj...@debrunners.com>]

Oyvind.Bakksjo@sun.com wrote:

> Daniel John Debrunner wrote:
> 
>> Oyvind.Bakksjo@Sun.COM wrote:
>>
>>
>>> I've been looking at implementing Statement.cancel(), and later,
>>> Statement.setQueryTimeout().
>>
>>
>>
>> So, quick summary, that method is not suitable for finding the correct
>> StatementContext from EmbedStatement.cancel().
>>
>> I'll look into how I think you should track down the StatementContext.
>> Dan.
> 
> 
> Hi Dan,
> 
> have you had any time to look into this?
Not a lot :-(

I've been thinking about it and there are a couple of issues:

1) StatementContexts are created dynamically for the current running
statement

2) Derby does not keep references to application JDBC objects below the
JDBC layer. This is to ensure when any JDBC object (e.g. Statement)
moves out of scope for the application, it will be garbage collected and
lead to its closure.

I wonder if from a StatementContext you can determine the activation
because that would provide the link to the Statement.


Dan.

Re: Implementing Statement.cancel()

[Oy...@sun.com]

Daniel John Debrunner wrote:
> Oyvind.Bakksjo@Sun.COM wrote:
> 
> 
>>I've been looking at implementing Statement.cancel(), and later,
>>Statement.setQueryTimeout().
> 
> 
> So, quick summary, that method is not suitable for finding the correct
> StatementContext from EmbedStatement.cancel().
> 
> I'll look into how I think you should track down the StatementContext.
> Dan.

Hi Dan,

have you had any time to look into this?

-- 
Øyvind Bakksjø
Sun Microsystems, Web Services, Database Technology Group
Haakon VII gt. 7b, N-7485 Trondheim, Norway
Tel: x43419 / +47 73842119, Fax: +47 73842101

Re: Implementing Statement.cancel()

[Shreyas Kaushik <Sh...@Sun.COM>]

Daniel John Debrunner wrote:

>Oyvind.Bakksjo@Sun.COM wrote:
>
>  
>
>>I've been looking at implementing Statement.cancel(), and later,
>>Statement.setQueryTimeout().
>>    
>>
>
><snip: good progress on implementation leading to this question>
>
>  
>
>>What I have been testing so far is using the
>>LanguageConnectionContext.getStatementContext() method through the lcc
>>reference available to both EmbedStatement and BasicNoPutResultSet.
>>However, it seems odd to me to go through a "connection context" object
>>to get the statement context. I'm not sure this is the right way to do
>>it. What if, for instance, there are multiple open statements on the
>>same connection? What statement's context object is returned?
>>    
>>
>
>That method will return the currently active top statement context, the
>statement being executed in the engine at the current time.
>
>If the application has multiple JDBC statements open then only one of
>them is active, ie. activly executing in the engine at any time, thus
>LanguageConnectionContext.getStatementContext() will return a
>StatementContext related to that execution.
>
>However, if the SQL statement being executed by the application's JDBC
>statement contains server side logic (triggers, procedures or
>functions), then the statement context may represent any SQL statement
>executed by the server logic.
>
>E.g. application has three open statements (Statement,
>PreparedStatements or CallableStatements)
>
>S1 prepared but no active ResultSet
>S2 executed, open ResultSet, application processing rows
>S3 executed, open ResultSet, application processing rows
>
>If the application has control, then there is no active
>StatementContext, since no active execution for that Connection is
>happening.
>
>Application executes S3.next()which takes some time
>
>Between the start of this S3.next() method and it returning, the engine
>is actively executing that statement. During this time no other activity
>for that Connection can be handled by the engine, it is blocked by
>synchronization at the JDBC level. That is if S2.next(), or S1.execute()
>is called, they will block on the java synchronization.
>
>At this point the active top StatementConxtext returned by
>LanguageConnectionContext.getStatementContext() will correspond to S3.
>
>But if S3's SQL statement included a function that in turn executed a
>SQL Statement using server-side JDBC (statement S4 using
>jdbc:default:connection), then while S4 was executing (S4.execute() or
>next() etc. for its ResultSet), the active top StatementContext would
>correspond to S4.
>
>So, quick summary, that method is not suitable for finding the correct
>StatementContext from EmbedStatement.cancel().
>
>I'll look into how I think you should track down the StatementContext.
>  
>
Dan,

Any pointers here ?

~ Shreyas

>Dan.
>
>
>  
>

Re: Implementing Statement.cancel()

[Oy...@Sun.COM]

Daniel John Debrunner skrev:
> Oyvind.Bakksjo@Sun.COM wrote:

>>[...] using the
>>LanguageConnectionContext.getStatementContext() method through the lcc
>>reference available to both EmbedStatement and BasicNoPutResultSet.
>>[...] What statement's context object is returned?
> 
> 
> That method will return the currently active top statement context, the
> statement being executed in the engine at the current time.

<details snipped>

> So, quick summary, that method is not suitable for finding the correct
> StatementContext from EmbedStatement.cancel().

That was my gut feeling, yes. :)

Thanks for a clear and enlightening answer.

> I'll look into how I think you should track down the StatementContext.
> Dan.

Thanks, that would be great.

-- 
Øyvind Bakksjø
Sun Microsystems, Web Services, Database Technology Group
Haakon VII gt. 7b, N-7485 Trondheim, Norway
Tel: x43419 / +47 73842119, Fax: +47 73842101

Re: Implementing Statement.cancel()

[Daniel John Debrunner <dj...@debrunners.com>]

Oyvind.Bakksjo@Sun.COM wrote:

> I've been looking at implementing Statement.cancel(), and later,
> Statement.setQueryTimeout().

<snip: good progress on implementation leading to this question>

> What I have been testing so far is using the
> LanguageConnectionContext.getStatementContext() method through the lcc
> reference available to both EmbedStatement and BasicNoPutResultSet.
> However, it seems odd to me to go through a "connection context" object
> to get the statement context. I'm not sure this is the right way to do
> it. What if, for instance, there are multiple open statements on the
> same connection? What statement's context object is returned?

That method will return the currently active top statement context, the
statement being executed in the engine at the current time.

If the application has multiple JDBC statements open then only one of
them is active, ie. activly executing in the engine at any time, thus
LanguageConnectionContext.getStatementContext() will return a
StatementContext related to that execution.

However, if the SQL statement being executed by the application's JDBC
statement contains server side logic (triggers, procedures or
functions), then the statement context may represent any SQL statement
executed by the server logic.

E.g. application has three open statements (Statement,
PreparedStatements or CallableStatements)

S1 prepared but no active ResultSet
S2 executed, open ResultSet, application processing rows
S3 executed, open ResultSet, application processing rows

If the application has control, then there is no active
StatementContext, since no active execution for that Connection is
happening.

Application executes S3.next()which takes some time

Between the start of this S3.next() method and it returning, the engine
is actively executing that statement. During this time no other activity
for that Connection can be handled by the engine, it is blocked by
synchronization at the JDBC level. That is if S2.next(), or S1.execute()
is called, they will block on the java synchronization.

At this point the active top StatementConxtext returned by
LanguageConnectionContext.getStatementContext() will correspond to S3.

But if S3's SQL statement included a function that in turn executed a
SQL Statement using server-side JDBC (statement S4 using
jdbc:default:connection), then while S4 was executing (S4.execute() or
next() etc. for its ResultSet), the active top StatementContext would
correspond to S4.

So, quick summary, that method is not suitable for finding the correct
StatementContext from EmbedStatement.cancel().

I'll look into how I think you should track down the StatementContext.
Dan.