You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Kathey Marsden <km...@sbcglobal.net> on 2006/04/04 00:00:14 UTC
Disussion surrounding holding off metadata changes until certain
issues resolved (was: Re: [jira] Updated: (DERBY-1107) For existing databases
JDBC metadata queries do not get updated properly between maintenance versions.)
Satheesh Bandaram wrote:
>Kathey Marsden wrote:
>
>
>
>>Satheesh Bandaram wrote:
>>
>>
>>I think on this one, we just need to get the DERBY-1076 patch
>>committed.
>>
>not sure if adding of new tests should stop development
>that passes current Derby checkin criteria of clean derbyAll.
>
>
>
I think if an area has regressions, significant quality issues, poor
test coverage or is a significant risk area it is appropriate for
anyone in the community to raise objections to new functionality being
added to that area until the core quality issues are addressed.
Upgrade is particularly sensitive because if not working, the resolution
may require time travel to resolve. Since there was no infrastructure in
place to handle upgrade changes of metadata, it had outstanding
regressions, and it has traditionally been an area of serious issue I
asked Knut to wait and he agreed., probably because he agrees with the
six core principals of the "Apache Way" especially the one about
"consistently high quality software".
http://www.apache.org/foundation/how-it-works.html
>>The blockage was based on the fact that we once had an irreconcilable
>>downgrade bug and seemed to be set up for that again.
>>
>>
>What was the code change that caused this problem in 5.1? You have
>provided symptoms of the problem. It would be good to understand the
>code change that cause this symptom.
>
>
>
The code change was that in 5.1.60 there were new classes and methods
that were added and referred to in the metadata SQL queries. Going
back to the previous version, the metadata queries were still there but
the classes and methods were not, so it caused this failure. In the
10.1 codeline, this means that no new functions or syntax not
compatible with older versions can be added to the queries. Adding the
drop on version change except for read only databases seems the safest
thing here to me. It seems like it will be a long time before we have a
10.2 release, who knows what will go in the 10.1 codeline.
>Again, I am not sure if DERBY-1107 should stop metadata developments.
>Wasn't this "issue" present from the beginning?
>
>
>
It is a serious outstanding issue with the handling of metadata
changes. It would be good to see it resolved soon, because it is again
one possibly needing time travel to resolve, but isn't stopping any
metadata changes from going in now.
Kathey
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by Kathey Marsden <km...@sbcglobal.net>.
Dag H. Wanvik wrote:
>Kathey Marsden <km...@sbcglobal.net> writes:
>
>
>
>>Once confirmed, could you file a bug against Network Server for this? It
>>should negotiate down properly.
>>
>>
>
>Will do.
>
>
>
>>For now it would be good for JCC to take the same path as the 10.1.1.0
>>client takes to correct this issue.
>>
>>
>
>Not sure what you have in mind here..
>
>
>
I don't have any particular solution in mind. All I am saying is that
whatever solution is used to fix the 10.2 Server so that it works with
10.1.1.0 client should also work for JCC.
Kathey
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by "Dag H. Wanvik" <Da...@Sun.COM>.
Kathey Marsden <km...@sbcglobal.net> writes:
> Once confirmed, could you file a bug against Network Server for this? It
> should negotiate down properly.
Will do.
> For now it would be good for JCC to take the same path as the 10.1.1.0
> client takes to correct this issue.
Not sure what you have in mind here..
Dag
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by Kathey Marsden <km...@sbcglobal.net>.
Dag H. Wanvik wrote:
>>>>>>"Kathey" == Kathey Marsden <km...@sbcglobal.net> wrote:
>>>>>>
>>>>>>
>
>Kathey> This is a regression for JCC, but is not a problem for the original
>Kathey> client release with the 10.2 server? I don't understand why JCC and the
>Kathey> original client 10.1.1.0 release would behave differently in this
>Kathey> regard. Could you explain?
>
>I'll try :) With an old (10.1) client and a new (10.2) server, there
>will be a problem for *both JCC and the Derby client* in that these
>four metadata calls (changed for SUR) will show a wrong value, since
>no down-negotiation happens for metadata calls:
>
> deletesAreDetected(TYPE_SCROLL_INSENSITIVE) -> true
> updatedAreDetected(TYPE_SCROLL_INSENSITIVE) -> true
> ownDeletesAreVisible(TYPE_SCROLL_INSENSITIVE) -> true
> ownUpdatesAreVisible(TYPE_SCROLL_INSENSITIVE) -> true
>
>
>
Once confirmed, could you file a bug against Network Server for this? It
should negotiate down properly.
[snip]
>FThat leaves the case of a (potential) new JCC driver and and old
>server. I won't address that.
>
>
>
For now it would be good for JCC to take the same path as the 10.1.1.0
client takes to correct this issue.
Since that potential JCC release does not exist yet.
Kathey
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by "Dag H. Wanvik" <Da...@Sun.COM>.
>>>>> "Kathey" == Kathey Marsden <km...@sbcglobal.net> wrote:
Kathey> This is a regression for JCC, but is not a problem for the original
Kathey> client release with the 10.2 server? I don't understand why JCC and the
Kathey> original client 10.1.1.0 release would behave differently in this
Kathey> regard. Could you explain?
I'll try :) With an old (10.1) client and a new (10.2) server, there
will be a problem for *both JCC and the Derby client* in that these
four metadata calls (changed for SUR) will show a wrong value, since
no down-negotiation happens for metadata calls:
deletesAreDetected(TYPE_SCROLL_INSENSITIVE) -> true
updatedAreDetected(TYPE_SCROLL_INSENSITIVE) -> true
ownDeletesAreVisible(TYPE_SCROLL_INSENSITIVE) -> true
ownUpdatesAreVisible(TYPE_SCROLL_INSENSITIVE) -> true
For the other method affected by SUR, supportsResultSetType, JCC
always returns false for all arguments, with both 10.1 and 10.2
servers, as far as I can see, and thus suffers no regression for this
call. The old Derby client has a bug (DERBY-965) which is only fixed
in 10.2, so the old Derby client would not show a regression with a
new server for supportsResultSetType.
Now, with new clients and new server, the Derby client will show the
SUR values for these calls, which is correct, including for the
soft/hard upgrade scenarios (after DERBY-1176). I don't know if there
will be a 10.2 version of JCC; if there will be, my point is that JCC
would continue to show wrong values for these four (detectability,
visibility) calls unless some special precaution was taken, since the
server don't differentiate between the clients on the metadata
calls. If the JCC client code for supportsResultSetType were corrected
to show the values returned by the server (in stead of always
returning false), it would have the same vulnerability: it would show
that SUR is supported, which I assume it would not be with JCC.
For the last case, new client and old server, for the Derby client the
four detectability and visibility calls are correct (return
false). The supportsResultSetType calls are wrong due to the server
side bugs of DERBY-965 still present in the old server, although now
manifested in a different way than with an old client, since the new
client correctly decodes the values transmitted:
SupportsResultSetConcurrency: TYPE_FORWARD_ONLY,CONCUR_READ_ONLY: false
SupportsResultSetConcurrency: TYPE_FORWARD_ONLY,CONCUR_UPDATABLE: true
SupportsResultSetConcurrency: TYPE_SCROLL_INSENSITIVE,CONCUR_READ_ONLY: false
SupportsResultSetConcurrency: TYPE_SCROLL_INSENSITIVE,CONCUR_UPDATABLE: true
SupportsResultSetConcurrency: TYPE_SCROLL_SENSITIVE,CONCUR_READ_ONLY: true
SupportsResultSetConcurrency: TYPE_SCROLL_SENSITIVE,CONCUR_UPDATABLE: true
Wrong, of course. The old client returned false for all of these.
That leaves the case of a (potential) new JCC driver and and old
server. I won't address that.
I hope this clarified matters in stead of muddling them further, the
details are a bit messy. Don't hesitate to ask if you have further
questions.
Dag
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by Kathey Marsden <km...@sbcglobal.net>.
Dag H. Wanvik wrote:
>K
>
>
>However, for the four other metadata calls that have changed with SUR:
>
> deletesAreDetected(TYPE_SCROLL_INSENSITIVE) -> true
> updatedAreDetected(TYPE_SCROLL_INSENSITIVE) -> true
> ownDeletesAreVisible(TYPE_SCROLL_INSENSITIVE) -> true
> ownUpdatesAreVisible(TYPE_SCROLL_INSENSITIVE) -> true
>
>we are not so lucky. On trying this for 10.2 trunk I see that JCC
>returns true for these as well (both in soft upgrade and and after
>hard upgrade, thanks to DERBY-1176), which *is* a regression for JCC
>apps:
>
>
>
This is a regression for JCC, but is not a problem for the original
client release with the 10.2 server? I don't understand why JCC and the
original client 10.1.1.0 release would behave differently in this
regard. Could you explain?
Thanks
Kathey
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by "Dag H. Wanvik" <Da...@Sun.COM>.
Kathey Marsden <km...@sbcglobal.net> writes:
> Thank you so much for calling out this potential compatibility issue in
> this thread. DERBY-775 was not on my compatibility radar.
>
> I tried the repro for DERBY-965 with JCC and the trunk server and see
> that JCC returns false these values for supportsResultSetConcurrency, so
> I think that we are ok for JCC too. Is that correct?
I have run this test again, adding the JCC client, after DERBY-775 and
DERBY-1176 went it. The JCC client return false for
SupportsResultSetConcurrency in all cases (10.1, 10.2 in soft upgrade
mode and after 10.2 hard upgrade). So I can see no regression for JCC
here. But the answer is wrong, of course ;)
However, for the four other metadata calls that have changed with SUR:
deletesAreDetected(TYPE_SCROLL_INSENSITIVE) -> true
updatedAreDetected(TYPE_SCROLL_INSENSITIVE) -> true
ownDeletesAreVisible(TYPE_SCROLL_INSENSITIVE) -> true
ownUpdatesAreVisible(TYPE_SCROLL_INSENSITIVE) -> true
we are not so lucky. On trying this for 10.2 trunk I see that JCC
returns true for these as well (both in soft upgrade and and after
hard upgrade, thanks to DERBY-1176), which *is* a regression for JCC
apps:
(
com.ibm.db2.jcc.DB2Driver:
SupportsResultSetConcurrency: TYPE_FORWARD_ONLY,CONCUR_READ_ONLY: false
SupportsResultSetConcurrency: TYPE_FORWARD_ONLY,CONCUR_UPDATABLE: false
SupportsResultSetConcurrency: TYPE_SCROLL_INSENSITIVE,CONCUR_READ_ONLY: false
SupportsResultSetConcurrency: TYPE_SCROLL_INSENSITIVE,CONCUR_UPDATABLE: false
SupportsResultSetConcurrency: TYPE_SCROLL_SENSITIVE,CONCUR_READ_ONLY: false
SupportsResultSetConcurrency: TYPE_SCROLL_SENSITIVE,CONCUR_UPDATABLE: false
deletesAreDetected: true
updatedAreDetected: true
ownDeletesAreVisible: true
ownUpdatesAreVisible: true
So I guess the JCC people should be alerted to this fact.
Dag
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by Kathey Marsden <km...@sbcglobal.net>.
Dag H. Wanvik wrote:
>>Old client, new server scenario: In this case, the downgrade of the
>>result set happens on the client, so applications would not be able
>>to use SUR. Only the new client lifts the restriction.
>>
>>Due to the bug in the 10.1 client code for
>>supportsResultSetConcurrency (DERBY-965), the client would still get
>>a "false" on this query, which is correct. Note that this is be
>>accident rather than by design! Derby has a weakness here in that
>>metadata queries are answered by the server without regard to the
>>client's version (capabilities).
>>
>>
Thank you so much for calling out this potential compatibility issue in
this thread. DERBY-775 was not on my compatibility radar.
I tried the repro for DERBY-965 with JCC and the trunk server and see
that JCC returns false these values for supportsResultSetConcurrency, so
I think that we are ok for JCC too. Is that correct?
com.ibm.db2.jcc.DB2Driver:
SupportsResultSetConcurrency: TYPE_FORWARD_ONLY,CONCUR_READ_ONLY: false
SupportsResultSetConcurrency: TYPE_FORWARD_ONLY,CONCUR_UPDATABLE: false
SupportsResultSetConcurrency: TYPE_SCROLL_INSENSITIVE,CONCUR_READ_ONLY:
false
SupportsResultSetConcurrency: TYPE_SCROLL_INSENSITIVE,CONCUR_UPDATABLE:
false
SupportsResultSetConcurrency: TYPE_SCROLL_SENSITIVE,CONCUR_READ_ONLY: false
SupportsResultSetConcurrency: TYPE_SCROLL_SENSITIVE,CONCUR_UPDATABLE: false
Apache Derby Network Server - 10.2.0.0 alpha shutdown at 2006-04-09
15:38:51.040 GMT
>Generally, it seems results of such metadata calls should potentially
>be "negotiated down" if the client is older and cannot support the new
>feature.. I am not sure of the machinery of NetDatabaseMetaData can be
>used as is for this purpose...(i.e. the serverSupport<FeatureX>
>methods). Or?
>
>
>
Right, I think there is still a hole here in terms of metadata handling
and how to negotiate down. I am guessing that will become incumbent on
someone who finds that they need that negotiation for some new
feature. It looks like you are "off the hook" this time because of
DERBY-965.
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by "Dag H. Wanvik" <Da...@Sun.COM>.
Kathey Marsden <km...@sbcglobal.net> writes:
>>As I was working with DERBY-775, it struck me that the JCC client
>>might get problems when metadata_net.properties is updated to reflect
>>Derby client capabilities not necessarily present in the JCC driver.
>>Should there perhaps be different code paths for the two drivers(JCC,
>>DerbyNetClient) wrt SYSIBM metadata? Not my itch, but...
>
> I am not clear on the specific issue you are referring to, but such
The metadata methods in question are supportsResultSetConcurrency,
ownDeletesAreVisible, ownUpdatesAreVisible, deletesAreDetected and
updatesAreDetected. With client SUR (not yet committed), they now
return true for scrollable insensitive result set types.
> compatibility issues would I think also exist when running with the
> original client release, Derby 10.1 and the new server. No ?
Yes and no, cf the explanation from the DERBY-775 writeup:
> Old client, new server scenario: In this case, the downgrade of the
> result set happens on the client, so applications would not be able
> to use SUR. Only the new client lifts the restriction.
>
> Due to the bug in the 10.1 client code for
> supportsResultSetConcurrency (DERBY-965), the client would still get
> a "false" on this query, which is correct. Note that this is be
> accident rather than by design! Derby has a weakness here in that
> metadata queries are answered by the server without regard to the
> client's version (capabilities).
>
> For the detectability metadata calls, the returned values would be
> wrong, but since this is new functionality it should be ok.
The changed value of
supportsResultSetConcurrency(TYPE_SCROLL_INSENSITIVE,
CONCUR_UPDATABLE), now true with the advent of SUR, could in theory
affect JCC client apps, but not 10.1 Derby network driver clients apps
due to the serendipty of DERBY-965: the driver will continue to return
false, as long as the patch for 965 is not back ported to the 10.1
branch.
Generally, it seems results of such metadata calls should potentially
be "negotiated down" if the client is older and cannot support the new
feature.. I am not sure of the machinery of NetDatabaseMetaData can be
used as is for this purpose...(i.e. the serverSupport<FeatureX>
methods). Or?
Back next week. One week easter vacation, have fun, all!
Dag
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by Kathey Marsden <km...@sbcglobal.net>.
Dag H. Wanvik wrote:
>Knut Anders Hatlen <Kn...@Sun.COM> writes:
>
>
>
>>Kathey Marsden <km...@sbcglobal.net> writes:
>>
>>
>>
>>>Rick Hillegas wrote:
>>>
>>>
>>>
>>>>So, I'm unclear: Are you still blocking metadata changes or are you
>>>>satisfied with the analysis done so far by Knut Anders and Dyre? Can
>>>>we proceed with metadata checkins provided that:
>>>>
>>>>1) Such patches clearly describe acceptable upgrade behavior
>>>>2) Clean upgrade test results accompany the submissions
>>>>
>>>>
>>>>
>>>I think that this would be fine and in addition would hope that any
>>>metadata queries changed/added were covered as part of the ugprade test.
>>>
>>>
>>I think it would be good if the upgrade test also ran metadata tests
>>in the DerbyNetClient framework. The only known upgrade issue in 10.2
>>is DERBY-1176, but it is not exposed by the upgrade test since it is
>>only seen in client/server mode.
>>
>>
>
>It would be nice it that were added.
>
>Client SUR changes some metadata (in metadata_net.properties) and
>relies on DERBY-1176 for correct operation under upgrade for some
>metadata queries, cf. the DERBY-775 write-up.
>
>As I was working with DERBY-775, it struck me that the JCC client
>might get problems when metadata_net.properties is updated to reflect
>Derby client capabilities not necessarily present in the JCC driver.
>Should there perhaps be different code paths for the two drivers(JCC,
>DerbyNetClient) wrt SYSIBM metadata? Not my itch, but...
>
>
>
I am not clear on the specific issue you are referring to, but such
compatibility issues would I think also exist when running with the
original client release, Derby 10.1 and the new server. No ?
Kathey
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by "Dag H. Wanvik" <Da...@Sun.COM>.
Knut Anders Hatlen <Kn...@Sun.COM> writes:
> Kathey Marsden <km...@sbcglobal.net> writes:
>
>> Rick Hillegas wrote:
>>
>>> So, I'm unclear: Are you still blocking metadata changes or are you
>>> satisfied with the analysis done so far by Knut Anders and Dyre? Can
>>> we proceed with metadata checkins provided that:
>>>
>>> 1) Such patches clearly describe acceptable upgrade behavior
>>> 2) Clean upgrade test results accompany the submissions
>>>
>> I think that this would be fine and in addition would hope that any
>> metadata queries changed/added were covered as part of the ugprade test.
>
> I think it would be good if the upgrade test also ran metadata tests
> in the DerbyNetClient framework. The only known upgrade issue in 10.2
> is DERBY-1176, but it is not exposed by the upgrade test since it is
> only seen in client/server mode.
It would be nice it that were added.
Client SUR changes some metadata (in metadata_net.properties) and
relies on DERBY-1176 for correct operation under upgrade for some
metadata queries, cf. the DERBY-775 write-up.
As I was working with DERBY-775, it struck me that the JCC client
might get problems when metadata_net.properties is updated to reflect
Derby client capabilities not necessarily present in the JCC driver.
Should there perhaps be different code paths for the two drivers(JCC,
DerbyNetClient) wrt SYSIBM metadata? Not my itch, but...
Dag
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by Knut Anders Hatlen <Kn...@Sun.COM>.
Kathey Marsden <km...@sbcglobal.net> writes:
> Rick Hillegas wrote:
>
>> So, I'm unclear: Are you still blocking metadata changes or are you
>> satisfied with the analysis done so far by Knut Anders and Dyre? Can
>> we proceed with metadata checkins provided that:
>>
>> 1) Such patches clearly describe acceptable upgrade behavior
>> 2) Clean upgrade test results accompany the submissions
>>
> I think that this would be fine and in addition would hope that any
> metadata queries changed/added were covered as part of the ugprade test.
I think it would be good if the upgrade test also ran metadata tests
in the DerbyNetClient framework. The only known upgrade issue in 10.2
is DERBY-1176, but it is not exposed by the upgrade test since it is
only seen in client/server mode.
--
Knut Anders
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by Rick Hillegas <Ri...@Sun.COM>.
Thanks, Kathey. This seems reasonable to me.
Regards,
-Rick
Kathey Marsden wrote:
>Rick Hillegas wrote:
>
>
>
>>So, I'm unclear: Are you still blocking metadata changes or are you
>>satisfied with the analysis done so far by Knut Anders and Dyre? Can
>>we proceed with metadata checkins provided that:
>>
>>1) Such patches clearly describe acceptable upgrade behavior
>>2) Clean upgrade test results accompany the submissions
>>
>>
>>
>I think that this would be fine and in addition would hope that any
>metadata queries changed/added were covered as part of the ugprade test.
>
>Kathey
>
>
>
>
>
>
>
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by Kathey Marsden <km...@sbcglobal.net>.
Rick Hillegas wrote:
> So, I'm unclear: Are you still blocking metadata changes or are you
> satisfied with the analysis done so far by Knut Anders and Dyre? Can
> we proceed with metadata checkins provided that:
>
> 1) Such patches clearly describe acceptable upgrade behavior
> 2) Clean upgrade test results accompany the submissions
>
I think that this would be fine and in addition would hope that any
metadata queries changed/added were covered as part of the ugprade test.
Kathey
Re: Disussion surrounding holding off metadata changes until certain
issues resolved
Posted by Rick Hillegas <Ri...@Sun.COM>.
>http://www.apache.org/foundation/how-it-works.html
>
>
>
>>>The blockage was based on the fact that we once had an irreconcilable
>>>downgrade bug and seemed to be set up for that again.
>>>
>>>
>>>
>>>
>>What was the code change that caused this problem in 5.1? You have
>>provided symptoms of the problem. It would be good to understand the
>>code change that cause this symptom.
>>
>>
>>
>>
>>
>The code change was that in 5.1.60 there were new classes and methods
>that were added and referred to in the metadata SQL queries. Going
>back to the previous version, the metadata queries were still there but
>the classes and methods were not, so it caused this failure. In the
>10.1 codeline, this means that no new functions or syntax not
>compatible with older versions can be added to the queries. Adding the
>drop on version change except for read only databases seems the safest
>thing here to me. It seems like it will be a long time before we have a
>10.2 release, who knows what will go in the 10.1 codeline.
>
>
>
I don't believe that the blocked patches have this defect. Am I wrong?
>>Again, I am not sure if DERBY-1107 should stop metadata developments.
>>Wasn't this "issue" present from the beginning?
>>
>>
>>
>>
>>
>It is a serious outstanding issue with the handling of metadata
>changes. It would be good to see it resolved soon, because it is again
>one possibly needing time travel to resolve, but isn't stopping any
>metadata changes from going in now.
>
>
>
So, I'm unclear: Are you still blocking metadata changes or are you
satisfied with the analysis done so far by Knut Anders and Dyre? Can we
proceed with metadata checkins provided that:
1) Such patches clearly describe acceptable upgrade behavior
2) Clean upgrade test results accompany the submissions
Regards,
-Rick