DERBY-2226: Bizarre "describe" behavior in ij?

[Army <qo...@gmail.com>]

If I run ij with the latest trunk and try to use the "describe" command on a 
database that was created in the 10.3 trunk prior to svn #496645, I get an error 
(NoSuchMethodError).  Revision 496645 was part of DERBY-2226.

My questions are:

  1) Does anyone else see this peculiar behavior?
  2) Does this matter?  If I create the database with 10.2 and then connect from 
the latest 10.3 the describe command works as expected.  So maybe this is just a 
side-effect from working on the development branch and there's nothing more to 
worry about...?

Just curious,
Army

----

Steps to reproduce:

 > svn update -r 496644
 > ant all
 > java -Dij.protocol=jdbc:derby: org.apache.derby.tools.ij
ij version 10.3
ij> connect 'hmm;create=true';
ij> create table tc (c char);
0 rows inserted/updated/deleted
ij> describe tc;
COLUMN_NAME         |TYPE_NAME|DEC&|NUM&|COLUM&|COLUMN_DEF|CHAR_OCTE&|IS_NULL&
------------------------------------------------------------------------------
C                   |CHAR     |NULL|NULL|1     |NULL      |2         |YES

1 row selected
ij> exit;
 > svn update -r 496645
 > ant clobber
 > ant all
 > java -Dij.protocol=jdbc:derby: org.apache.derby.tools.ij
ij> connect 'hmm';
ij> describe tc;
ERROR XJ001: Java exception: 
'org.apache.derby.iapi.sql.execute.ResultSetFactory: method 
getIndexRowToBaseRowResultSet(JILorg/apache/derby/iapi/sql/execute/NoPutResultSet;Lorg/apache/derby/iapi/services/loader/GeneratedMethod;ILjava/lang/String;IIILorg/apache/derby/iapi/services/loade: 
java.lang.NoSuchMethodError'.

Re: DERBY-2226: Bizarre "describe" behavior in ij?

[Daniel John Debrunner <dj...@apache.org>]

Bryan Pendleton wrote:
>> I appologize for all the problems I seem to have created
> 
> I don't think you did anything wrong, and you shouldn't feel
> bad. We all learned something from this, and it's a quite
> subtle thing. The code appears to be operating as designed, and
> your change was sound; it's just a bumpy road when you're
> operating with the daily head of the trunk.
> 
> I think it's cool that Army noticed this, and cool that Kristian
> was able to confirm it, and it seems that the explanation of
> the code's behavior matches all the known cases (thanks Dan for
> explaining it so clearly!).
> 
> I think episodes like this provide a chance for us all to learn
> something that would have been quite painful to find out on our
> own, and I don't think anybody should feel bad about the situation.

+1 - well put Bryan.

Dan.

Re: DERBY-2226: Bizarre "describe" behavior in ij?

[Bryan Pendleton <bp...@amberpoint.com>]

> I appologize for all the problems I seem to have created

I don't think you did anything wrong, and you shouldn't feel
bad. We all learned something from this, and it's a quite
subtle thing. The code appears to be operating as designed, and
your change was sound; it's just a bumpy road when you're
operating with the daily head of the trunk.

I think it's cool that Army noticed this, and cool that Kristian
was able to confirm it, and it seems that the explanation of
the code's behavior matches all the known cases (thanks Dan for
explaining it so clearly!).

I think episodes like this provide a chance for us all to learn
something that would have been quite painful to find out on our
own, and I don't think anybody should feel bad about the situation.

thanks,

bryan

Re: DERBY-2226: Bizarre "describe" behavior in ij?

[Dy...@Sun.COM]

Daniel John Debrunner <dj...@apache.org> writes:

> Army wrote:
>> If I run ij with the latest trunk and try to use the "describe"
>> command on a database that was created in the 10.3 trunk prior to
>> svn #496645, I get an error (NoSuchMethodError).  Revision 496645
>> was part of DERBY-2226.
>> My questions are:
>>  1) Does anyone else see this peculiar behavior?
>>  2) Does this matter?  If I create the database with 10.2 and then
>> connect from the latest 10.3 the describe command works as expected.
>> So maybe this is just a side-effect from working on the development
>> branch and there's nothing more to worry about...?
>
> What is happening is the stored compiled plan for the metadata query
> was compiled against a set of classes that have since changed their
> signature. Derby does handle this if the engine's release number does
> not match the release the database was last booted as, e.g. a change
> from 10.2.1.5 to 10.2.1.6 will force recompiling of stored plans. This
> is how changes in a branch are handled.
>
> I don't think it matters, the development trunk is just that a
> development trunk. Trying to ensure the current trunk worked against
> any previous version of a database created by the development trunk I
> think is too much work. Handling upgrade at defined boundaries is hard
> enough :-)

I appologize for all the problems I seem to have created for
people. If I had foreseen that this would be a problem I would have
posted a warning to derby-dev. I'll try to do that in the future... :(

-- 
dt

Re: DERBY-2226: Bizarre "describe" behavior in ij?

[Daniel John Debrunner <dj...@apache.org>]

Army wrote:
> If I run ij with the latest trunk and try to use the "describe" command 
> on a database that was created in the 10.3 trunk prior to svn #496645, I 
> get an error (NoSuchMethodError).  Revision 496645 was part of DERBY-2226.
> 
> My questions are:
> 
>  1) Does anyone else see this peculiar behavior?
>  2) Does this matter?  If I create the database with 10.2 and then 
> connect from the latest 10.3 the describe command works as expected.  So 
> maybe this is just a side-effect from working on the development branch 
> and there's nothing more to worry about...?

What is happening is the stored compiled plan for the metadata query was 
compiled against a set of classes that have since changed their 
signature. Derby does handle this if the engine's release number does 
not match the release the database was last booted as, e.g. a change 
from 10.2.1.5 to 10.2.1.6 will force recompiling of stored plans. This 
is how changes in a branch are handled.

I don't think it matters, the development trunk is just that a 
development trunk. Trying to ensure the current trunk worked against any 
previous version of a database created by the development trunk I think 
is too much work. Handling upgrade at defined boundaries is hard enough :-)

Dan.

Re: DERBY-2226: Bizarre "describe" behavior in ij?

[Daniel John Debrunner <dj...@apache.org>]

Army wrote:
> Thank you for the reply, Kristian.  Dan also replied to my email with an 
> explanation of what was happening to cause this error (thanks, 
> Dan!)--maybe he answered your questions in the process?
> 
>> In my case, the failing statement was a trigger. My guess is that the 
>> trigger was compiled against the old code (happened to be 10.2, but 
>> could be anything before r496645 I think), and when it is "retrieved" 
>> and executed the constructor it is calling no longer exist (signature 
>> has changed).
> 
> Based on Dan's description of the problem I would think that a 10.2 
> trigger should still succeed with the latest trunk (because there is a 
> version difference and thus the trigger's SPS should get 
> recompiled)--unless I'm misunderstanding something there?  Are you sure 
> the trigger that is failing was a 10.2 trigger and not a 10.3 (pre svn 
> #496645) one?

Just to be clear it's not when a trigger (or any stored plan) was 
originally created, but the version of database engine when the stored 
plan was last compiled. Any boot of an engine will mark all stored plans 
as needing recompile if the engine version is different to the last 
engine that booted the version.

So one potential for Kristian's scenario is:

1) create trigger on 10.2

2) boot with 10.3.0.0 (pre 496645) and call trigger
    (trigger will be recompiled)

3) boot with 10.3.0.0 (post 496645) and call trigger
    (trigger will not recompiled but relies on code signatures pre-496645).

Dan.

Re: DERBY-2226: Bizarre "describe" behavior in ij?

[Army <qo...@gmail.com>]

Thank you for the reply, Kristian.  Dan also replied to my email with an 
explanation of what was happening to cause this error (thanks, Dan!)--maybe he 
answered your questions in the process?

> In my case, the failing statement was a trigger. My guess is that the 
> trigger was compiled against the old code (happened to be 10.2, but 
> could be anything before r496645 I think), and when it is "retrieved" 
> and executed the constructor it is calling no longer exist (signature 
> has changed).

Based on Dan's description of the problem I would think that a 10.2 trigger 
should still succeed with the latest trunk (because there is a version 
difference and thus the trigger's SPS should get recompiled)--unless I'm 
misunderstanding something there?  Are you sure the trigger that is failing was 
a 10.2 trigger and not a 10.3 (pre svn #496645) one?

> And, does the "describe" command also use a trigger/SPS/something such 
> that my error scenario can be said to be the same as Army's problem?

As Dan mentioned the "describe" command uses a metadata query which in turn has 
a stored compiled plan.  So yes, I think your scenario is probably similar to 
the one I was seeing.

Thanks again to you and Dan for your replies!

Army

Re: DERBY-2226: Bizarre "describe" behavior in ij?

[Kristian Waagan <Kr...@Sun.COM>]

Army skrev:
> If I run ij with the latest trunk and try to use the "describe" 
> command on a database that was created in the 10.3 trunk prior to svn 
> #496645, I get an error (NoSuchMethodError).  Revision 496645 was part 
> of DERBY-2226.
>
> My questions are:
>
>  1) Does anyone else see this peculiar behavior?
Hello Army,

I just saw this today, but in a totally different scenario. I was 
running a J2EE application, and saw the same error message as you. After 
a little investigation, I also landed on DERBY-2226 as the culprit.
In my case, the failing statement was a trigger. My guess is that the 
trigger was compiled against the old code (happened to be 10.2, but 
could be anything before r496645 I think), and when it is "retrieved" 
and executed the constructor it is calling no longer exist (signature 
has changed).

>  2) Does this matter?  If I create the database with 10.2 and then 
> connect from the latest 10.3 the describe command works as expected.  
> So maybe this is just a side-effect from working on the development 
> branch and there's nothing more to worry about...?
This I don't know. I can't see this being added as part of the patch. Is 
there a general mechanism for updating triggers when upgrading Derby?
And, does the "describe" command also use a trigger/SPS/something such 
that my error scenario can be said to be the same as Army's problem?


-- 
Kristian
>
> Just curious,
> Army
>
> ----
>
> Steps to reproduce:
>
> > svn update -r 496644
> > ant all
> > java -Dij.protocol=jdbc:derby: org.apache.derby.tools.ij
> ij version 10.3
> ij> connect 'hmm;create=true';
> ij> create table tc (c char);
> 0 rows inserted/updated/deleted
> ij> describe tc;
> COLUMN_NAME         
> |TYPE_NAME|DEC&|NUM&|COLUM&|COLUMN_DEF|CHAR_OCTE&|IS_NULL&
> ------------------------------------------------------------------------------ 
>
> C                   |CHAR     |NULL|NULL|1     |NULL      |2         |YES
>
> 1 row selected
> ij> exit;
> > svn update -r 496645
> > ant clobber
> > ant all
> > java -Dij.protocol=jdbc:derby: org.apache.derby.tools.ij
> ij> connect 'hmm';
> ij> describe tc;
> ERROR XJ001: Java exception: 
> 'org.apache.derby.iapi.sql.execute.ResultSetFactory: method 
> getIndexRowToBaseRowResultSet(JILorg/apache/derby/iapi/sql/execute/NoPutResultSet;Lorg/apache/derby/iapi/services/loader/GeneratedMethod;ILjava/lang/String;IIILorg/apache/derby/iapi/services/loade: 
> java.lang.NoSuchMethodError'.
>