derby standards

[Rick Hillegas <Ri...@sun.com>]

I'm afraid I'm having a hard time figuring out Derby's relationship to 
what might be called standards. The following principles seem to be 
regularly advocated:

1) Derby should avoid inventing its own syntax and apis.

2) Instead, Derby should adopt syntax and apis endorsed by the following 
authorities:

  - ANSI SQL 2003
  - JDBC 4.0
  - JSRs
  - Other Apache projects

 From time to time, other authorities are recommended even though they 
conflict with principle (2):

3) Non-ANSI syntax used by popular databases like Oracle, DB2, Postgres, 
and MySQL.

4) Constraints imposed by DRDA.

Principle (3) proves to be particularly nettlesome since the popular 
databases often disagree. As we expand Derby, I would like to understand 
how we reconcile these principles. Perhaps, first, we should state what 
Derby hopes to gain by compliance. The following benefits might apply:

A) Familiar syntax and apis encourage developers to use Derby for new 
embedded applications.

B) Compatible syntax and apis encourage migration of old applications to 
Derby from other databases.

C) Compatible syntax and apis make it easy to scale up usage of a 
Derby-developed application by migrating it to an enterprise-calibre dbms.

With four popular databases to keep in mind, benefits (B) and (C) seem 
hard to satisfy. I am worried that we are in danger of building a 
database which does not deliver any of these benefits. I would like to 
see us clarify our goals and compliance policies.

Cheers,
-Rick

Re: derby standards

[Francois Orsini <fr...@gmail.com>]

I do agree with a lot of this *however* and as we all pretty much
know, many of the ANSI SQL items have been made standard after certain
RDBMS vendors pushed for them to become a standard...It's great to be
innovative and come up with enhanced features and syntaxes as long as
IMHO _if_  1) it is obvious and makes sense to invent and propose them
and there is no existing standard, 2) there is an actual follow-up and
representation at some ANSI committee...Basically, if one decides to
come up with a new syntax or important enhancement to the SQL world,
then it better makes sense and be proposed (follow-up) for ANSI
acceptance...

If you look back at how "stored procedure" and "trigger" made it to
ANSI specs amongst other additions, I'm glad the innovative RDBMS
vendor(s) came up with these items in the first place ;) - sure enough
and eventhough there is now some ANSI standard based stored procedure
and trigger, existing RDBMS vendors will continue to support their
current extensions (i.e. T-SQL, PL/SQL) - not just because they have
to be backward compatible with the existing customer base but also
because the extensions are (currently) richer than the ANSI standard
base...

It is important to respect the existing and defined standards - We
need to make the extra effort to support and reinforce standards
compliance in Derby as Rick mentioned - completely agree with this -
this is the absolute minimum IMO. Now providing extensions on top of
the standard (base) is something that also makes a database
attractive... ;) but if you want portable code, then use the least
common denominator which is ANSI SQL and other official standards...

I recall hearing about ANSI SQL compliance tests being available
somewhere...not sure where this stands...JDBC used to have some as
well..

--francois

Re: derby standards

[David Van Couvering <Da...@Sun.COM>]

Others here at Sun have mentioned the same point to me, it allows for 
third-party clients.  So I withdraw my claim that it is not a 
"published" API.

David

Daniel John Debrunner wrote:

>David Van Couvering wrote:
>
>  
>
>>I do want to make one point. In my opinion, the DRDA communication
>>between client and server is purely an internal interface. It is not
>>touched by our users and changing the underlying protocol would in no
>>way impact the portability of applications written in Derby to other
>>databases.  I think in the past the value of using DRDA is it allows the
>>IBM unversal driver to work with Derby; when Derby was an internal DB at
>>IBM this made a lot of sense.  My question is is this still a
>>requirement for Derby as an Apache project?  At this point, I see no
>>compelling reason to move away from DRDA, but if at some point it
>>becomes clear it really is hamstringing us in some way, either in terms
>>of performance, security or functionality, I'd like to understand where
>>we as a community stand on this question.  Do we try to convince the
>>Open Group to change DRDA every time we have an issue, or do we forge
>>ahead with our own modifications even if they're non-standard?
>>    
>>
>
>
>In a way DRDA is an api into Derby. Derby's use of a standard protocol
>allows third parties to write clients that connect to Derby. There are
>two concrete examples of these, IBM's JDBC and ODBC drivers. Now I agree
>that those exist due to Derby's history, but there are other companies
>out there that write such clients.
>
>Totally throwing out DRDA seems to be a bad idea because we have to
>replace it with something, and why define a new database protocol when
>one exists already. Adopting another could be a possibility, I looked at
>Sybase's TDS given the OpenTDS project but since that is reverse
>engineering and has lots of cases like 'byte[4] no idea what goes here'
>it didn't seem attractive to me.
>
>It seems unlikely that DRDA will not be sufficient for our performance
>needs, DB2 reguarly holds the top spots in TPC-C and other benchmarks,
>so DRDA seems to be up to the task.
>
>I agree its functionality it may fall short, but Rick's current approach
>seems great, carefully define how we want to expand it, document it and
>try to get it into the OpenGroup.
>
>Dan.
>
>
>  
>

Re: derby standards

[Daniel John Debrunner <dj...@debrunners.com>]

David Van Couvering wrote:

> I do want to make one point. In my opinion, the DRDA communication
> between client and server is purely an internal interface. It is not
> touched by our users and changing the underlying protocol would in no
> way impact the portability of applications written in Derby to other
> databases.  I think in the past the value of using DRDA is it allows the
> IBM unversal driver to work with Derby; when Derby was an internal DB at
> IBM this made a lot of sense.  My question is is this still a
> requirement for Derby as an Apache project?  At this point, I see no
> compelling reason to move away from DRDA, but if at some point it
> becomes clear it really is hamstringing us in some way, either in terms
> of performance, security or functionality, I'd like to understand where
> we as a community stand on this question.  Do we try to convince the
> Open Group to change DRDA every time we have an issue, or do we forge
> ahead with our own modifications even if they're non-standard?


In a way DRDA is an api into Derby. Derby's use of a standard protocol
allows third parties to write clients that connect to Derby. There are
two concrete examples of these, IBM's JDBC and ODBC drivers. Now I agree
that those exist due to Derby's history, but there are other companies
out there that write such clients.

Totally throwing out DRDA seems to be a bad idea because we have to
replace it with something, and why define a new database protocol when
one exists already. Adopting another could be a possibility, I looked at
Sybase's TDS given the OpenTDS project but since that is reverse
engineering and has lots of cases like 'byte[4] no idea what goes here'
it didn't seem attractive to me.

It seems unlikely that DRDA will not be sufficient for our performance
needs, DB2 reguarly holds the top spots in TPC-C and other benchmarks,
so DRDA seems to be up to the task.

I agree its functionality it may fall short, but Rick's current approach
seems great, carefully define how we want to expand it, document it and
try to get it into the OpenGroup.

Dan.

Re: derby standards

[David Van Couvering <Da...@Sun.COM>]

I do want to make one point. In my opinion, the DRDA communication 
between client and server is purely an internal interface. It is not 
touched by our users and changing the underlying protocol would in no 
way impact the portability of applications written in Derby to other 
databases.  I think in the past the value of using DRDA is it allows the 
IBM unversal driver to work with Derby; when Derby was an internal DB at 
IBM this made a lot of sense.  My question is is this still a 
requirement for Derby as an Apache project?  At this point, I see no 
compelling reason to move away from DRDA, but if at some point it 
becomes clear it really is hamstringing us in some way, either in terms 
of performance, security or functionality, I'd like to understand where 
we as a community stand on this question.  Do we try to convince the 
Open Group to change DRDA every time we have an issue, or do we forge 
ahead with our own modifications even if they're non-standard?

I agree I am a little confused about the use of non-standard "industry 
standard" feature like SELECT FOR UPDATE.  Another one that comes to 
mind is the DESCRIBE mechanism that many databases have but which is non 
standard.

Thanks,

David

Rick Hillegas wrote:

> I'm afraid I'm having a hard time figuring out Derby's relationship to 
> what might be called standards. The following principles seem to be 
> regularly advocated:
>
> 1) Derby should avoid inventing its own syntax and apis.
>
> 2) Instead, Derby should adopt syntax and apis endorsed by the 
> following authorities:
>
>  - ANSI SQL 2003
>  - JDBC 4.0
>  - JSRs
>  - Other Apache projects
>
> From time to time, other authorities are recommended even though they 
> conflict with principle (2):
>
> 3) Non-ANSI syntax used by popular databases like Oracle, DB2, 
> Postgres, and MySQL.
>
> 4) Constraints imposed by DRDA.
>
> Principle (3) proves to be particularly nettlesome since the popular 
> databases often disagree. As we expand Derby, I would like to 
> understand how we reconcile these principles. Perhaps, first, we 
> should state what Derby hopes to gain by compliance. The following 
> benefits might apply:
>
> A) Familiar syntax and apis encourage developers to use Derby for new 
> embedded applications.
>
> B) Compatible syntax and apis encourage migration of old applications 
> to Derby from other databases.
>
> C) Compatible syntax and apis make it easy to scale up usage of a 
> Derby-developed application by migrating it to an enterprise-calibre 
> dbms.
>
> With four popular databases to keep in mind, benefits (B) and (C) seem 
> hard to satisfy. I am worried that we are in danger of building a 
> database which does not deliver any of these benefits. I would like to 
> see us clarify our goals and compliance policies.
>
> Cheers,
> -Rick

Re: derby standards

[Kathey Marsden <km...@sbcglobal.net>]

Rick Hillegas wrote:

> I'm afraid I'm having a hard time figuring out Derby's relationship to
> what might be called standards. The following principles seem to be
> regularly advocated:
>
> 1) Derby should avoid inventing its own syntax and apis.
>
> 2) Instead, Derby should adopt syntax and apis endorsed by the
> following authorities:
>
>  - ANSI SQL 2003
>  - JDBC 4.0
>  - JSRs
>  - Other Apache projects
>
> From time to time, other authorities are recommended even though they
> conflict with principle (2):
>
> 3) Non-ANSI syntax used by popular databases like Oracle, DB2,
> Postgres, and MySQL.
>
> 4) Constraints imposed by DRDA.
>
> Principle (3) proves to be particularly nettlesome since the popular
> databases often disagree. As we expand Derby, I would like to
> understand how we reconcile these principles. Perhaps, first, we
> should state what Derby hopes to gain by compliance. The following
> benefits might apply:
>
> A) Familiar syntax and apis encourage developers to use Derby for new
> embedded applications.
>
> B) Compatible syntax and apis encourage migration of old applications
> to Derby from other databases.
>
> C) Compatible syntax and apis make it easy to scale up usage of a
> Derby-developed application by migrating it to an enterprise-calibre
> dbms.
>
I feel a bit like the Lorax speaking for the trees, but here goes....

In the case of existing syntax, I don't think we should remove or even
deprecate it in a lot of instances,  but it would be great to  introduce
a standard alternative.     I think it important to consider the
installed  user base and the advantage keeping the current syntax
working  provides.

D)   Users find their applications still work when they upgrade.


Kathey