Defining compatibility stability (was Re: Server and client compatibility test for 10.2 and 10.1)

["David W. Van Couvering" <Da...@Sun.COM>]

Thanks for trackin these down, Dag.

So, I'm not sure how this works.  We have made compatible changes in 
terms of the API, but incompatible in terms of the output, particularly 
of message strings.  Is this considered a failure, and we are no longer 
backward-compatible?

I agree we want to be backward-compatible, but do we need to apply the 
same level of rigor to ij output format and error message strings as we 
do to APIs?

At Sun we have a mechanism for declaring the stability of each interface 
we present to users, where interfaces include but are not limited to:

- APIs
- command-line interface name and syntax
- stdout
- stderr
- error codes from command-line interfaces
- jar file names
- package names
- directory structure
- install location
- configuration file names
- configuration file syntax and format
- database file format
- transaction log file format
- error log file format

Each stability level implies a contract as to how and when an interface 
can change (generally: it can change at a major release, at a minor 
release, or at any time).  Generally things like the API and CLI syntax 
have a higher stability guarantee than say stderr or the error log format.

I think it might be worth our while to put up a Wiki page and identify 
all our interfaces and the level of stability we want to have for them, 
so we know what we should be testing for in terms of compatibility 
between releases...

David

Dag H. Wanvik wrote:
> Hi,
> 
> 
>>>>>>"Kathey" == Kathey Marsden <km...@sbcglobal.net> wrote:
> 
> 
> Kathey> >Server: 10.1, Client: 10.2, derbyTesting.jar: 10.1
> Kathey> >---------------------------------------------------------------------
> Kathey> >jvms: jdk15, ibm142
> Kathey> >
> Kathey> > Test failures:
> Kathey> >derbynetclientmats/derbynetmats/derbynetmats.fail:jdbcapi/dbMetaDataJdbc30.java
> Kathey> >derbynetclientmats/derbynetmats/derbynetmats.fail:jdbcapi/metadata.java
> Kathey> >derbynetclientmats/derbynetmats/derbynetmats.fail:jdbcapi/odbc_metadata.java
> Kathey> >derbynetclientmats/derbynetmats/derbynetmats.fail:lang/forupdate.sql
> Kathey> >derbynetclientmats/derbynetmats/derbynetmats.fail:lang/updatableResultSet.java
> Kathey> >derbynetclientmats/derbynetmats/derbynetmats.fail:store/holdCursorJDBC30.sql
> Kathey> >derbynetclientmats/derbynetmats/derbynetmats.fail:jdbcapi/LOBTest.java
> Kathey> >derbynetclientmats/derbynetmats/derbynetmats.fail:jdbcapi/metadataJdbc20.java
> Kathey> >derbynetclientmats/derbynetmats/derbynetmats.fail:jdbcapi/connectionJdbc20.java
> 
> I have run this mixed release scenario (details below) and checked the
> results for:
> 
> 	lang/forupdate.sql
> 	lang/updatableResultSet.java. 
> 
> In both cases, the differences are due to changes in the client, and
> thus expected. The changes are reflected in the current (10.2) client
> masters. Mostly, the diffs are changed exception messages, and in some
> cases, SQLStates.
> 
> Dag
> 
> ******* Start Suite: derbynetclientmats 2006-02-01 02:41:16 *******
> ------------------ Java Information ------------------
> Java Version:    1.5.0_04
> Java Vendor:     Sun Microsystems Inc.
> Java home:       /usr/local/java/jdk1.5.0_04/jre
> Java classpath:  /home/dw136774/derby/trunk/jars/sane/derbyclient.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derby.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbytools.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbynet.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbyTesting.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbyLocale_de_DE.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbyLocale_es.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbyLocale_fr.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbyLocale_it.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbyLocale_ja_JP.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbyLocale_ko_KR.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbyLocale_pt_BR.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbyLocale_zh_CN.jar:/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbyLocale_zh_TW.j
ar:/home/dw136774/derby/trunk/tools/java/jakarta-oro-2.0.8.jar
> OS name:         SunOS
> OS architecture: x86
> OS version:      5.10
> Java user name:  dw136774
> Java user home:  /home/dw136774
> Java user dir:   /home/dw136774/derbytests/compat2
> java.specification.name: Java Platform API Specification
> java.specification.version: 1.5
> --------- Derby Information --------
> JRE - JDBC: J2SE 5.0 - JDBC 3.0
> [/home/dw136774/derby/trunk/jars/sane/derbyclient.jar] 10.2.0.0 alpha - (371922M)
> [/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derby.jar] 10.1.2.1 - (330608)
> [/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbytools.jar] 10.1.2.1 - (330608)
> [/home/dw136774/derby-10.1/db-derby-10.1.2.1-lib/lib/derbynet.jar] 10.1.2.1 - (330608)