You are viewing a plain text version of this content. The canonical link for it is here.
Posted to axis-tck@ws.apache.org by Sam Ruby <ru...@apache.org> on 2002/08/02 15:45:20 UTC

Automating the TCKs - comments and questions

I've been trying to automate the execution of the TCKs against Axis,
starting with SAAJ.  I've attached several files to this e-mail based on
this effort.

* saaj_feedback.html describes the environment in which I ran the tess 
and then notes several issues with the latest ReleaseNotes-saaj.html,
virtually all minor clarifications with the possible exception of the
one in the configuring the tomcat server.

* saajtck.sh is my implementation of the installation instructions.
Feel free to review for correctness or completeness.

* failures.saaj contain a list of tests that fail for me using the
latest nightly build version of Axis.  26/283 fail.  As Dims previously
was able to pass this test, my working assumption at this point is that
there is either an environment issue on my end or a recent regression
introduced into Axis.

My real purpose for sending this note is to see if we can brainstorm on
how I can completely automate this test.  It seems to me that all the
infrastructure is in place for me to request that all the tests be run
simply by specifying a work directory and a configuration (ts_unix).
However, the only interface I am aware of at the moment to do this is a
GUI.  Is there a command line interface to this functionallity available?

Note: I am quite willing to launch the gui every time I get a new rev of
a TCK, produce and capture configuration files (such as default.jti).
What I desire is the ability to automatically rerun such tests against
the latest version of the software being tested.

Any ideas you might have would be appreciated.

- Sam Ruby

Re: Publishing the results of the TCKs

Posted by Sam Ruby <ru...@apache.org>.
Jim Driscoll wrote:
> 
> Unless, of course, Apache were willing to certify non-members as being
> covered by the license, which you aren't willing to do - so it's a
> problem created by both sides - Sun's desire for confidentiality, and
> Apache's reluctance to vouch for all her committers.

Vouch for what?  I can understand the argument that Apache needs to 
protect access to the TCK itself, but what is the argument for 
protectings committers from knowing when their changes break 
compatiblitity with the JSR?

Can't we find some middle ground here?

- Sam Ruby


Re: Publishing the results of the TCKs

Posted by Jim Driscoll <ji...@sun.com>.
Sam Ruby wrote:
> 
> Jim Driscoll wrote:
>  > Hi Sam -
>  >
>  > I checked with the lawyers, and this was one of the things the is
>  > specifically not allowed by the contract.
> 
> This is very frustrating as you and I specifically talked about doing
> exactly this prior to the Apache/Sun agreement.
> 
> But somehow, this is not surprising, as it completes the cycle.  Apache
> sabre rattles.  There is much tension.  Agreement is reached just before
> the deadline.  Yet when it comes time for execution, the story is the
> same "my lawyer won't let me do it."

Actually, it's "your contract won't let you do it".  I'm sorry, but I
don't remember the conversation - I'm assuming it was before the
signing, where things went back on forth a bit.  But I talked with so
many people about this.  If I misled you in any way, or gave you
incorrect information, I'm truly sorry.  I do remember discussing this
at length with Jason, Chuck and a variety of lawyers.  In the end, the
confidentiality provisions were weakened, but not eliminated.  I
personally would have liked them eliminated, but there were good
arguments for allowing them to remain.

I assume you guys all discussed this at length internally as well, but I
was excluded from those discussions, so I couldn't really say.

>  > You may publish it on a password protected website, or email it to
>  > either jcp@apache.org (all Apache members) or axis-tck@sml.apache.org
>  > (who all have rights to view the TCK either from Apache or
>  > elsewhere), but the sole thing you can say about TCK test status
>  > publicly is "passed" or "failed".
> 
> Gee, thanks.

Figured you'd be thrilled :-(
 
>  > Allow me to explain the reasoning here:  the thought is when you
>  > allow people to publish full test results, you introduce the notion
>  > of "mostly passing", i.e. "We pass 550 TCK tests out of 600!", when
>  > really, the status is "Failed the TCK".
>  >
>  > Yes, I know that that isn't Apache's intent in any way.  But the
>  > contract's purpose is to guard against bad intent...
>  >
>  > Hope that this isn't too noxious.
> 
> First thing I would like to clearly establish is that this is a
> requirement of Sun and/or the leads for these specifications, and not of
> the JCP itself, right?  After all, IIRC the business terms for JSR 110
> are as follows:

<snip>

Yep, it's a requirement of the contract signed between Sun and Apache. 
AFAIK, it's not a JCP requirement.  However, I'm not the JCP expert. 
Jason / Chuck are more likely to have the definitive answer there, or I
can check internally if they're unavailable.  I'd have to refer to the
JSPA verbiage, and that's still a work in progress anyway.

> Now let me explain what I was trying to accomplish.  At the present
> time, we pass 100% of the SAA JTCK, and are within two tests of passing
> the JAXRPC TCK.  What I was hoping to do was to set things up so that
> the tests ran automatically and generally successfully.  But at the
> point where somebody - any contributor, really - broke compatibility in
> a nightly build, they would be notified of that fact, generally within
> 24 hours.
> 
> Instead, what is being proposed is that unless they are one of the
> select few, not only can't they run these tests themselves, they can't
> even directly view the results of their efforts.
> 
> So, yes, I view that as noxious.

Unless, of course, Apache were willing to certify non-members as being
covered by the license, which you aren't willing to do - so it's a
problem created by both sides - Sun's desire for confidentiality, and
Apache's reluctance to vouch for all her committers.

One solution that I can think of is to have the people who are active
committers apply to the board for license permission.  Also noxious, but
it does what you want, eventually.

BTW - congratulations on the team's progress!  Almost there.

Jim

Re: Publishing the results of the TCKs

Posted by Sam Ruby <ru...@apache.org>.
Jim Driscoll wrote:
 > Hi Sam -
 >
 > I checked with the lawyers, and this was one of the things the is
 > specifically not allowed by the contract.

This is very frustrating as you and I specifically talked about doing
exactly this prior to the Apache/Sun agreement.

But somehow, this is not surprising, as it completes the cycle.  Apache
sabre rattles.  There is much tension.  Agreement is reached just before
the deadline.  Yet when it comes time for execution, the story is the
same "my lawyer won't let me do it."

 > You may publish it on a password protected website, or email it to
 > either jcp@apache.org (all Apache members) or axis-tck@sml.apache.org
 > (who all have rights to view the TCK either from Apache or
 > elsewhere), but the sole thing you can say about TCK test status
 > publicly is "passed" or "failed".

Gee, thanks.

 > Allow me to explain the reasoning here:  the thought is when you
 > allow people to publish full test results, you introduce the notion
 > of "mostly passing", i.e. "We pass 550 TCK tests out of 600!", when
 > really, the status is "Failed the TCK".
 >
 > Yes, I know that that isn't Apache's intent in any way.  But the
 > contract's purpose is to guard against bad intent...
 >
 > Hope that this isn't too noxious.

First thing I would like to clearly establish is that this is a
requirement of Sun and/or the leads for these specifications, and not of
the JCP itself, right?  After all, IIRC the business terms for JSR 110
are as follows:

 > The TCK will be open source and be released with the Common Public
 > License (CPL). Making it open increases the community understanding
 > of the API since the source provides more visibility to the test
 > coverage. Availability of the source will help reference implementers
 > understand the test requirements better. Over time this will lead to
 > a better implementation. Even though the source is open, there is a
 > maintainer of the official TCK, the JSR Maintenance Spec Lead. Open
 > Source allows others to 'submit' changes to the TCK, but doesn't
 > guarantee that they are approved. Compliance will only be by passing
 > the TCK approved by the committee appropriate to the approved API.
 > Compliance will be claimed via self-certification. In a true open
 > source environment, customers, analysts, consultants, governmental
 > agencies, etc. will all have access to the TCK to see how it works
 > and run implementations through it. Violators of the
 > self-certification and logo use rules will be quickly exposed. This
 > "Market Driven" compliance is far more powerful than trying to
 > maintain control through onerous legal conditions and exceptions.
 > There are no separate business issues involved in providing JSR110
 > certification. Self-certification is the rule. Enforcement is simple.
 >  If after JSR110 is approved, someone claims compliance and any user
 > finds otherwise, the TCK provides a relatively objective way of
 > proving noncompliance."

Now let me explain what I was trying to accomplish.  At the present
time, we pass 100% of the SAA JTCK, and are within two tests of passing
the JAXRPC TCK.  What I was hoping to do was to set things up so that
the tests ran automatically and generally successfully.  But at the
point where somebody - any contributor, really - broke compatibility in 
a nightly build, they would be notified of that fact, generally within 
24 hours.

Instead, what is being proposed is that unless they are one of the
select few, not only can't they run these tests themselves, they can't
even directly view the results of their efforts.

So, yes, I view that as noxious.

 > Jim

- Sam Ruby


Re: Publishing the results of the TCKs

Posted by Jim Driscoll <ji...@sun.com>.
Hi Sam -

I checked with the lawyers, and this was one of the things the is
specifically not allowed by the contract.

You may publish it on a password protected website, or email it to
either jcp@apache.org (all Apache members) or axis-tck@sml.apache.org
(who all have rights to view the TCK either from Apache or elsewhere),
but the sole thing you can say about TCK test status publicly is
"passed" or "failed".

Allow me to explain the reasoning here:  the thought is when you allow
people to publish full test results, you introduce the notion of "mostly
passing", i.e. "We pass 550 TCK tests out of 600!", when really, the
status is "Failed the TCK".

Yes, I know that that isn't Apache's intent in any way.  But the
contract's purpose is to guard against bad intent...

Hope that this isn't too noxious.

Jim

Sam Ruby wrote:
> 
> I was able to find [1], so I produced [2].  At the moment, it is
> password protected (user=axis, passwd=thanksJason).  What I want to know
> if *ANYBODY* has *ANY* concerns with publically publishing (i.e., on an
> apache.org site with no password) *ANY_PART_OF* the output the TCK when
> run against Axis?
> 
> Speak now, or forever hold your peace...
> 
> - Sam Ruby
> 
> [1] http://developer.java.sun.com/developer/technicalArticles/JCPtools2/
> [2] http://intertwingly.net/saajtck/report.html



Publishing the results of the TCKs

Posted by Sam Ruby <ru...@apache.org>.
I was able to find [1], so I produced [2].  At the moment, it is 
password protected (user=axis, passwd=thanksJason).  What I want to know 
if *ANYBODY* has *ANY* concerns with publically publishing (i.e., on an 
apache.org site with no password) *ANY_PART_OF* the output the TCK when 
run against Axis?

Speak now, or forever hold your peace...

- Sam Ruby

[1] http://developer.java.sun.com/developer/technicalArticles/JCPtools2/
[2] http://intertwingly.net/saajtck/report.html


Re: Automating the TCKs - comments and questions

Posted by Lance Andersen <La...@sun.com>.
There is a way to run the tck from gnumake, but we would need to include 
the makefiles in the bundle.  I am checking on the possibility.  This 
would then allow you to kick off a cron job from the command line.

Note however for official certification, the suite(s) must be run from 
the gui.

Lance

Sam Ruby wrote:

> Sam Ruby wrote:
>
>>
>> * saajtck.sh is my implementation of the installation instructions.
>> Feel free to review for correctness or completeness.
>
>
> Dims called me, and suggested that I *not* copy the JAXM classes to 
> common/lib in order to test Axis.  Once I did this, the complete set 
> of tests passed.
>
> Now all I need is a way to automate the launching of these tests and I 
> can verify compliance nightly.
>
> - Sam Ruby
>



Re: Automating the TCKs - comments and questions

Posted by Sam Ruby <ru...@apache.org>.
Sam Ruby wrote:
> 
> * saajtck.sh is my implementation of the installation instructions.
> Feel free to review for correctness or completeness.

Dims called me, and suggested that I *not* copy the JAXM classes to 
common/lib in order to test Axis.  Once I did this, the complete set of 
tests passed.

Now all I need is a way to automate the launching of these tests and I 
can verify compliance nightly.

- Sam Ruby


Re: Automating the TCKs - comments and questions

Posted by Lance Andersen <La...@sun.com>.
I pushed to your private directory a version of the saaj and jaxrpc tck 
which contains the makefiles.  
 From the src tree, you can  type tsmake runclient to run the entire 
suite or a specific section of the tests starting at the directory level 
where you kickoff tsmake.

You will want to run the install script again or manually copy the files 
from the install directory.

regards,
Lance

Sam Ruby wrote:

> I've been trying to automate the execution of the TCKs against Axis,
> starting with SAAJ.  I've attached several files to this e-mail based on
> this effort.
>
> * saaj_feedback.html describes the environment in which I ran the tess 
> and then notes several issues with the latest ReleaseNotes-saaj.html,
> virtually all minor clarifications with the possible exception of the
> one in the configuring the tomcat server.
>
> * saajtck.sh is my implementation of the installation instructions.
> Feel free to review for correctness or completeness.
>
> * failures.saaj contain a list of tests that fail for me using the
> latest nightly build version of Axis.  26/283 fail.  As Dims previously
> was able to pass this test, my working assumption at this point is that
> there is either an environment issue on my end or a recent regression
> introduced into Axis.
>
> My real purpose for sending this note is to see if we can brainstorm on
> how I can completely automate this test.  It seems to me that all the
> infrastructure is in place for me to request that all the tests be run
> simply by specifying a work directory and a configuration (ts_unix).
> However, the only interface I am aware of at the moment to do this is a
> GUI.  Is there a command line interface to this functionallity available?
>
> Note: I am quite willing to launch the gui every time I get a new rev of
> a TCK, produce and capture configuration files (such as default.jti).
> What I desire is the ability to automatically rerun such tests against
> the latest version of the software being tested.
>
> Any ideas you might have would be appreciated.
>
> - Sam Ruby
>
> ------------------------------------------------------------------------
>
> Installing the required software:
>
>     J2SE 1.3.1_04-b02
>
>     Redhat Linux 7.3
>
>     Tomcat 4.0.4
>
>     java_xml_pack-summer-02
>
>     saaj_tck-1_1-fcs-bin-06_jun_2002 + saajtck-temp-a
>
> Setting Up your environment
>
>     2b) the name of the script is saajsetup (no .sh suffix)
>
>     4b) in JAXPAK, jaxp-api.jar is not in the same directory as
>     saaj-api.jar.  I ended up including jaxp-1.2/*.jar and
>     jaxm-1.1/lib/*.jar.
>
> Configuring the Tomcat server
>
>     Nowhere does it specify where you are supposed to place the
>     software you are trying to test.  I would presume that it needs to
>     go into common/lib so that it is accessible from each of the wars,
>     but it is not clear to me which SAAJ implementation would be
>     picked up if this were done as the instructions specify that
>     $SAAJ_HOME/lib/*.jar should also be copied there.
>
> Running the SAAJ tests
>
>     One needs to execute . ./ts_env first.  On many Unix systems, your
>     current directory is not by default in one's PATH, so a "./ts" is
>     required.
>
>  
>
>      
>
>      
>
>------------------------------------------------------------------------
>
>export AXIS_HOME=~/jakarta/xml-axis/java/build
>
>export TS_HOME=~/saajtck
>export JAXM_HOME=/opt/java_xml_pack-summer-02/jaxm-1.1
>export CATALINA_HOME=~/tomcat4
>export CATALINA_OPTS=-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog 
>export JAXP_HOME=/opt/java_xml_pack-summer-02/jaxp-1.2
>
>cd
>rm -rf $TS_HOME
>unzip -q ~/tck/saaj_tck-1_1-fcs-bin-06_jun_2002.zip
>unzip -q -o ~/tck/saajtck-temp-a.zip
>
>cd $TS_HOME/install
>sh saajsetup
>
>LOCALCLASSES=$CATALINA_HOME/common/lib/servlet.jar
>for i in $TS_HOME/lib/*.jar $JAXM_HOME/lib/*.jar $JAXP_HOME/*.jar; do
>  LOCALCLASSES=$LOCALCLASSES:$i
>done
>
>echo s!JAVA_HOME=!JAVA_HOME=$JAVA_HOME! > tsenv.sed
>echo s!LOCAL_CLASSES=!LOCAL_CLASSES=$LOCALCLASSES! >> tsenv.sed
>cp ../bin/ts_env ts_env.base
>sed -f tsenv.sed < ts_env.base > ../bin/ts_env
>
>cp ../bin/ts.jte ts_jte.base
>sed s/8080/5049/ < ts_jte.base > ../bin/ts.jte
>
>$CATALINA_HOME/bin/shutdown.sh
>cp $JAXM_HOME/lib/*.jar $CATALINA_HOME/common/lib
>cp $JAXM_HOME/jaxm/*.jar $CATALINA_HOME/common/lib
>cp $TS_HOME/lib/harness.jar $CATALINA_HOME/common/lib
>cp $TS_HOME/dist/*.war $CATALINA_HOME/webapps
>cp $AXIS_HOME/lib/axis.jar $AXIS_HOME/lib/wsdl4j.jar $CATALINA_HOME/common/lib
>$CATALINA_HOME/bin/startup.sh
>
>cd ..
>mkdir src/com/sun/ts/tests/axis
>mkdir src/com/sun/ts/tests/axis/jtData
>testsuite=src/com/sun/ts/tests/axis/jtData/testsuite
>echo #JavaTest Work Directory: Test Suite Info> $testsuite
>echo #`date`>> $testsuite
>echo root=`pwd`/src/com/sun/ts/tests>> $testsuite
>cp ~/tck/default.jti src/com/sun/ts/tests/axis/jtData
>
>cd bin
>. ./ts_env
>./ts
>cd ../src/com/sun/ts/tests/axis
>find saaj -name *.jtr -exec grep -l "^test result: Failed." {} \; > ~/failures.saaj
>echo `wc -l ~/failures.saaj`
>  
>
>------------------------------------------------------------------------
>
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getMimeHeader2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getAllMimeHeaders1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getAllMimeHeaders2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getAllMimeHeaders3Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getAllMimeHeaders4Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_setMimeHeader2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_removeMimeHeader1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_removeMimeHeader3Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_removeMimeHeader4Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getMatchingMimeHeaders5Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getNonMatchingMimeHeaders1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getNonMatchingMimeHeaders4Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getContentId3Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_setContent1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getContent1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getMimeHeader2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getAllMimeHeaders1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getAllMimeHeaders2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getAllMimeHeaders3Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getAllMimeHeaders4Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_setMimeHeader2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_removeMimeHeader1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_removeMimeHeader3Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_removeMimeHeader4Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getMatchingMimeHeaders5Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getNonMatchingMimeHeaders4Test_from_standalone.jtr
>  
>


Re: Automating the TCKs - comments and questions

Posted by Lance Andersen <La...@sun.com>.
Yeah I assume you are using the -batch option.  Having the makefiles is 
still a better option as it makes it easier to debug test problems when 
you have to rebuild.

Sam Ruby wrote:

> Lance Andersen wrote:
>
>>
>> Wanted to give you an update on this.  We will  push out a new 
>> versionof the tck so that you can run tcks via gnumake using our 
>> tsmake script.  Going forward we will move from gnumake to ant, but 
>> not for the current version of the tck.
>>
>> I will get this out to you tommorrow.
>>
>> Please note that for certification, you need to use the gui, but can 
>> use gnumake for your development and QA.
>
>
> Last Friday, I was able to find the answers I needed at 
> http://developer.java.sun.com/developer/technicalArticles/JCPtools2/ 
> and have been (successfully) running the automated SAAJTCK for days 
> now and am near the point where I can run the JAXRPCTCK successfully 
> with no human intervention.
>
> I have been posting the results to 
> http://www.intertwingly.net/saajtck/report.html and 
> http://www.intertwingly.net/jaxrpctck/report.html respectively.  User 
> is Axis.  Password is thanksJason.
>
> - Sam Ruby
>



Re: Automating the TCKs - comments and questions

Posted by Sam Ruby <ru...@intertwingly.net>.
Lance Andersen wrote:
> 
> Wanted to give you an update on this.  We will  push out a new versionof 
> the tck so that you can run tcks via gnumake using our tsmake script. 
>  Going forward we will move from gnumake to ant, but not for the current 
> version of the tck.
> 
> I will get this out to you tommorrow.
> 
> Please note that for certification, you need to use the gui, but can use 
> gnumake for your development and QA.

Last Friday, I was able to find the answers I needed at 
http://developer.java.sun.com/developer/technicalArticles/JCPtools2/ and 
have been (successfully) running the automated SAAJTCK for days now and 
am near the point where I can run the JAXRPCTCK successfully with no 
human intervention.

I have been posting the results to 
http://www.intertwingly.net/saajtck/report.html and 
http://www.intertwingly.net/jaxrpctck/report.html respectively.  User is 
Axis.  Password is thanksJason.

- Sam Ruby


Re: Automating the TCKs - comments and questions

Posted by Lance Andersen <La...@sun.com>.
Sam,

Wanted to give you an update on this.  We will  push out a new versionof 
the tck so that you can run tcks via gnumake using our tsmake script. 
 Going forward we will move from gnumake to ant, but not for the current 
version of the tck.

I will get this out to you tommorrow.

Please note that for certification, you need to use the gui, but can use 
gnumake for your development and QA.

Regards,
Lance

Sam Ruby wrote:

> I've been trying to automate the execution of the TCKs against Axis,
> starting with SAAJ.  I've attached several files to this e-mail based on
> this effort.
>
> * saaj_feedback.html describes the environment in which I ran the tess 
> and then notes several issues with the latest ReleaseNotes-saaj.html,
> virtually all minor clarifications with the possible exception of the
> one in the configuring the tomcat server.
>
> * saajtck.sh is my implementation of the installation instructions.
> Feel free to review for correctness or completeness.
>
> * failures.saaj contain a list of tests that fail for me using the
> latest nightly build version of Axis.  26/283 fail.  As Dims previously
> was able to pass this test, my working assumption at this point is that
> there is either an environment issue on my end or a recent regression
> introduced into Axis.
>
> My real purpose for sending this note is to see if we can brainstorm on
> how I can completely automate this test.  It seems to me that all the
> infrastructure is in place for me to request that all the tests be run
> simply by specifying a work directory and a configuration (ts_unix).
> However, the only interface I am aware of at the moment to do this is a
> GUI.  Is there a command line interface to this functionallity available?
>
> Note: I am quite willing to launch the gui every time I get a new rev of
> a TCK, produce and capture configuration files (such as default.jti).
> What I desire is the ability to automatically rerun such tests against
> the latest version of the software being tested.
>
> Any ideas you might have would be appreciated.
>
> - Sam Ruby
>
> ------------------------------------------------------------------------
>
> Installing the required software:
>
>     J2SE 1.3.1_04-b02
>
>     Redhat Linux 7.3
>
>     Tomcat 4.0.4
>
>     java_xml_pack-summer-02
>
>     saaj_tck-1_1-fcs-bin-06_jun_2002 + saajtck-temp-a
>
> Setting Up your environment
>
>     2b) the name of the script is saajsetup (no .sh suffix)
>
>     4b) in JAXPAK, jaxp-api.jar is not in the same directory as
>     saaj-api.jar.  I ended up including jaxp-1.2/*.jar and
>     jaxm-1.1/lib/*.jar.
>
> Configuring the Tomcat server
>
>     Nowhere does it specify where you are supposed to place the
>     software you are trying to test.  I would presume that it needs to
>     go into common/lib so that it is accessible from each of the wars,
>     but it is not clear to me which SAAJ implementation would be
>     picked up if this were done as the instructions specify that
>     $SAAJ_HOME/lib/*.jar should also be copied there.
>
> Running the SAAJ tests
>
>     One needs to execute . ./ts_env first.  On many Unix systems, your
>     current directory is not by default in one's PATH, so a "./ts" is
>     required.
>
>  
>
>      
>
>      
>
>------------------------------------------------------------------------
>
>export AXIS_HOME=~/jakarta/xml-axis/java/build
>
>export TS_HOME=~/saajtck
>export JAXM_HOME=/opt/java_xml_pack-summer-02/jaxm-1.1
>export CATALINA_HOME=~/tomcat4
>export CATALINA_OPTS=-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog 
>export JAXP_HOME=/opt/java_xml_pack-summer-02/jaxp-1.2
>
>cd
>rm -rf $TS_HOME
>unzip -q ~/tck/saaj_tck-1_1-fcs-bin-06_jun_2002.zip
>unzip -q -o ~/tck/saajtck-temp-a.zip
>
>cd $TS_HOME/install
>sh saajsetup
>
>LOCALCLASSES=$CATALINA_HOME/common/lib/servlet.jar
>for i in $TS_HOME/lib/*.jar $JAXM_HOME/lib/*.jar $JAXP_HOME/*.jar; do
>  LOCALCLASSES=$LOCALCLASSES:$i
>done
>
>echo s!JAVA_HOME=!JAVA_HOME=$JAVA_HOME! > tsenv.sed
>echo s!LOCAL_CLASSES=!LOCAL_CLASSES=$LOCALCLASSES! >> tsenv.sed
>cp ../bin/ts_env ts_env.base
>sed -f tsenv.sed < ts_env.base > ../bin/ts_env
>
>cp ../bin/ts.jte ts_jte.base
>sed s/8080/5049/ < ts_jte.base > ../bin/ts.jte
>
>$CATALINA_HOME/bin/shutdown.sh
>cp $JAXM_HOME/lib/*.jar $CATALINA_HOME/common/lib
>cp $JAXM_HOME/jaxm/*.jar $CATALINA_HOME/common/lib
>cp $TS_HOME/lib/harness.jar $CATALINA_HOME/common/lib
>cp $TS_HOME/dist/*.war $CATALINA_HOME/webapps
>cp $AXIS_HOME/lib/axis.jar $AXIS_HOME/lib/wsdl4j.jar $CATALINA_HOME/common/lib
>$CATALINA_HOME/bin/startup.sh
>
>cd ..
>mkdir src/com/sun/ts/tests/axis
>mkdir src/com/sun/ts/tests/axis/jtData
>testsuite=src/com/sun/ts/tests/axis/jtData/testsuite
>echo #JavaTest Work Directory: Test Suite Info> $testsuite
>echo #`date`>> $testsuite
>echo root=`pwd`/src/com/sun/ts/tests>> $testsuite
>cp ~/tck/default.jti src/com/sun/ts/tests/axis/jtData
>
>cd bin
>. ./ts_env
>./ts
>cd ../src/com/sun/ts/tests/axis
>find saaj -name *.jtr -exec grep -l "^test result: Failed." {} \; > ~/failures.saaj
>echo `wc -l ~/failures.saaj`
>  
>
>------------------------------------------------------------------------
>
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getMimeHeader2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getAllMimeHeaders1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getAllMimeHeaders2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getAllMimeHeaders3Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getAllMimeHeaders4Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_setMimeHeader2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_removeMimeHeader1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_removeMimeHeader3Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_removeMimeHeader4Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getMatchingMimeHeaders5Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getNonMatchingMimeHeaders1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getNonMatchingMimeHeaders4Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getContentId3Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_setContent1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/SOAPPart/URLClient_getContent1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getMimeHeader2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getAllMimeHeaders1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getAllMimeHeaders2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getAllMimeHeaders3Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getAllMimeHeaders4Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_setMimeHeader2Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_removeMimeHeader1Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_removeMimeHeader3Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_removeMimeHeader4Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getMatchingMimeHeaders5Test_from_standalone.jtr
>saaj/api/javax_xml_soap/AttachmentPart/URLClient_getNonMatchingMimeHeaders4Test_from_standalone.jtr
>  
>