[jira] Updated: (DERBY-658) test harness improvements required for running on non-ASCII systems

["Myrna van Lunteren (JIRA)" <de...@db.apache.org>]

     [ http://issues.apache.org/jira/browse/DERBY-658?page=all ]

Myrna van Lunteren updated DERBY-658:
-------------------------------------

    Attachment: DERBY-658_101_20060407_1.stat
                DERBY-658_101_20060407_1.diff

This - DERBY-658_101_20060407_1.* - is the first patch in this effort. This patch is for 10.1. It seemed to me a simple merge would not be possible, so I've duplicated this effort for main (That patch is forthcoming, need to sync up and rerun derbyall one more time).

This patch implements stesp 2-5. Note that step 1 is already implemented on the main codeline, I decided not to copy that code into 10.1 at this time.

With this change, we can no longer use 'run resource' in ij tests. IJ needs to run in the local encoding, the resource in derbyTesting.jar will be in UTF-8 encoding, so they cannot be used. This has impacted mainly on a number of .sql tests in tests/store. I've had to choose between making additional _app.properties files to specify the supportfiles, and modifying the default_app.properties. I chose for the latter, preferring fewer files. However, this then made it impossible for a number of _app.properties files to usedefaults.

Note that this change is a first step - after this, individual tests will need changes to pass on non-ISO-8559 systems...
Also, step 6 will need to get implemented at a later date - I'm even thinking we may not implement it on 10.1...

Note that we need this code in 10.1 to faciltate fixing of bugs on zOS  in 10.1


> test harness improvements required for running on non-ASCII systems
> -------------------------------------------------------------------
>
>          Key: DERBY-658
>          URL: http://issues.apache.org/jira/browse/DERBY-658
>      Project: Derby
>         Type: Bug

>   Components: Test
>     Versions: 10.1.1.0
>  Environment: all but especially testing should be done on zOS
>     Reporter: Myrna van Lunteren
>     Assignee: Myrna van Lunteren
>     Priority: Minor
>      Fix For: 10.2.0.0
>  Attachments: DERBY-658_101_20060407_1.diff, DERBY-658_101_20060407_1.stat
>
> The current functionTests test harness needs adjustment for running on non-ASCII systems like zOS.
> Currently, when using derbyTesting.jar built for instance on a windows or linux system on zOS the tests do not run, because the properties and runall files cannot be understood. 
> Until now, testers on zOS had to unjar derbyTesting.jar, then run native2ascii -Cp 1047 -reverse on all appropriate files (.sql, .txt, .out, .properties, .runall, .asc, .exclude, etc). 
> This is a labor intensive and error prone process. Furthermore, it causes test failures such as reported with DERBY-575, because tests may assume a certain file to be a specific length, which no longer holds true after the native2ascii conversion.
> The test harness should get modified to always read the files in the same encoding.
> Note however, that the comparison of actual .out and the master should still result in files readable on the local system, to enable a human to evaluate failures and results. At the same time, this raises the concern that someone might check-in an update to the master with an incorrect encoding.
> To resolve the main issue, I propose the following:
> -  Set the default encoding in the harness.
> - for each test 
> 1)  determine if the test encoding is set. We can probably use ij.ui.codeset - otherwise a new property is needed.
>      Note that this means that .properties, .runall and .exclude  files are always read in fixed/default encoding.
> 2) read the master/sql files in in the default/fixed encoding unless the encoding property is set for a test
> 3) Write the output out in the local encoding (the way is done currently) unless the encoding property is set (if set, write out in that encoding)
> 4) Change the code that creates tmpmstr to always apply instead of only for networkserver. tmpmstr files will be created in the local encoding.
> 5) Have FileCompare  read tmpmstr in in the local encoding for the comparison.
> 6) either document that test development/adjustment need to be at least be verified on an ascii system, or add another property that causes a copy of the actual output to be created in the default/fixed encoding.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira