You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@harmony.apache.org by "Alexander Kleymenov (JIRA)" <ji...@apache.org> on 2006/11/23 12:32:02 UTC
[jira] Created: (HARMONY-2283) [drlvm][regression] Regression
Testing Infrastructure proposal
[drlvm][regression] Regression Testing Infrastructure proposal
--------------------------------------------------------------
Key: HARMONY-2283
URL: http://issues.apache.org/jira/browse/HARMONY-2283
Project: Harmony
Issue Type: Improvement
Components: DRLVM
Reporter: Alexander Kleymenov
There is no infrastructure for automated regression testing in Harmony DRLVM project.
This JIRA report is intended to initiate the discussions and works on it.
With taking into account the specific of the product, the following requirements to infrastructure can be defined:
1. It should support tests in different formats (JUnit, pure Java, etc) and implemented in heterogeneous sources (Java, C/C++, Java assembler etc).
2. It should provide convenient means for test run configuration (parallel/sequential execution, execution timeouts, command line parameters specifying, environment variables presetting).
3. It should provide means for rapid and convenient integration of new test.
4. It should be based on commonly used means.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (HARMONY-2283) [drlvm][regression] Regression
Testing Infrastructure proposal
Posted by "Geir Magnusson Jr (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-2283?page=all ]
Geir Magnusson Jr closed HARMONY-2283.
--------------------------------------
> [drlvm][regression] Regression Testing Infrastructure proposal
> --------------------------------------------------------------
>
> Key: HARMONY-2283
> URL: http://issues.apache.org/jira/browse/HARMONY-2283
> Project: Harmony
> Issue Type: Improvement
> Components: DRLVM
> Reporter: Alexander Kleymenov
> Assigned To: Geir Magnusson Jr
> Attachments: patch-reg-tests.txt
>
>
> There is no infrastructure for automated regression testing in Harmony DRLVM project.
> This JIRA report is intended to initiate the discussions and works on it.
> With taking into account the specific of the product, the following requirements to infrastructure can be defined:
> 1. It should support tests in different formats (JUnit, pure Java, etc) and implemented in heterogeneous sources (Java, C/C++, Java assembler etc).
> 2. It should provide convenient means for test run configuration (parallel/sequential execution, execution timeouts, command line parameters specifying, environment variables presetting).
> 3. It should provide means for rapid and convenient integration of new test.
> 4. It should be based on commonly used means.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HARMONY-2283) [drlvm][regression] Regression
Testing Infrastructure proposal
Posted by "Geir Magnusson Jr (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-2283?page=all ]
Geir Magnusson Jr resolved HARMONY-2283.
----------------------------------------
Resolution: Fixed
r479764
Ubuntu 6.
Nice job
> [drlvm][regression] Regression Testing Infrastructure proposal
> --------------------------------------------------------------
>
> Key: HARMONY-2283
> URL: http://issues.apache.org/jira/browse/HARMONY-2283
> Project: Harmony
> Issue Type: Improvement
> Components: DRLVM
> Reporter: Alexander Kleymenov
> Assigned To: Geir Magnusson Jr
> Attachments: patch-reg-tests.txt
>
>
> There is no infrastructure for automated regression testing in Harmony DRLVM project.
> This JIRA report is intended to initiate the discussions and works on it.
> With taking into account the specific of the product, the following requirements to infrastructure can be defined:
> 1. It should support tests in different formats (JUnit, pure Java, etc) and implemented in heterogeneous sources (Java, C/C++, Java assembler etc).
> 2. It should provide convenient means for test run configuration (parallel/sequential execution, execution timeouts, command line parameters specifying, environment variables presetting).
> 3. It should provide means for rapid and convenient integration of new test.
> 4. It should be based on commonly used means.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HARMONY-2283) [drlvm][regression] Regression
Testing Infrastructure proposal
Posted by "Geir Magnusson Jr (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-2283?page=comments#action_12453441 ]
Geir Magnusson Jr commented on HARMONY-2283:
--------------------------------------------
I'm playing with it. Things I want to do :
1) make it so that the jasmin jar is optional, or make it so that it's fetched automatically, dependably
2) when running, h000 fails :
run-test:
[junit] Running org.apache.harmony.drlvm.tests.regression.h0000.DirectByteBufferTest
[junit] HMYEXEL062E Internal VM error: Failed to create Java VM
[junit] FAILED to invoke JVM.
Haven't looked into it yet.
> [drlvm][regression] Regression Testing Infrastructure proposal
> --------------------------------------------------------------
>
> Key: HARMONY-2283
> URL: http://issues.apache.org/jira/browse/HARMONY-2283
> Project: Harmony
> Issue Type: Improvement
> Components: DRLVM
> Reporter: Alexander Kleymenov
> Assigned To: Geir Magnusson Jr
> Attachments: patch-reg-tests.txt
>
>
> There is no infrastructure for automated regression testing in Harmony DRLVM project.
> This JIRA report is intended to initiate the discussions and works on it.
> With taking into account the specific of the product, the following requirements to infrastructure can be defined:
> 1. It should support tests in different formats (JUnit, pure Java, etc) and implemented in heterogeneous sources (Java, C/C++, Java assembler etc).
> 2. It should provide convenient means for test run configuration (parallel/sequential execution, execution timeouts, command line parameters specifying, environment variables presetting).
> 3. It should provide means for rapid and convenient integration of new test.
> 4. It should be based on commonly used means.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (HARMONY-2283) [drlvm][regression] Regression
Testing Infrastructure proposal
Posted by "Geir Magnusson Jr (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-2283?page=all ]
Geir Magnusson Jr reassigned HARMONY-2283:
------------------------------------------
Assignee: Geir Magnusson Jr
> [drlvm][regression] Regression Testing Infrastructure proposal
> --------------------------------------------------------------
>
> Key: HARMONY-2283
> URL: http://issues.apache.org/jira/browse/HARMONY-2283
> Project: Harmony
> Issue Type: Improvement
> Components: DRLVM
> Reporter: Alexander Kleymenov
> Assigned To: Geir Magnusson Jr
> Attachments: patch-reg-tests.txt
>
>
> There is no infrastructure for automated regression testing in Harmony DRLVM project.
> This JIRA report is intended to initiate the discussions and works on it.
> With taking into account the specific of the product, the following requirements to infrastructure can be defined:
> 1. It should support tests in different formats (JUnit, pure Java, etc) and implemented in heterogeneous sources (Java, C/C++, Java assembler etc).
> 2. It should provide convenient means for test run configuration (parallel/sequential execution, execution timeouts, command line parameters specifying, environment variables presetting).
> 3. It should provide means for rapid and convenient integration of new test.
> 4. It should be based on commonly used means.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HARMONY-2283) [drlvm][regression] Regression
Testing Infrastructure proposal
Posted by "Alexander Kleymenov (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-2283?page=all ]
Alexander Kleymenov updated HARMONY-2283:
-----------------------------------------
Attachment: patch-reg-tests.txt
Attached patch contains initial implementation of regression testing infrastructure. Apache Ant was chosen as a base for implementation because:
1. It is platform independent tool installed on every workstation used for Harmony/DRLVM development.
2. It provides all the means to build/execute "heterogeneous" tests.
3. It allows configuring of tests execution.
4. For default test format (such as JUnit) it allows automatic test building/executing (after the test source have been added to the test tree it will automatically be built and executed by build target)
5. For some sophisticated test demanding non-standard launching method it is possible to create special Ant target (possible in little ant file placed next to the test) describing the way of launching it. These targets can use common parameters and macrodefinitions defined for regression testing.
Several regression tests are provided as a usage example. They are placed in src/test/regression directory of DRLVM workspace. Subdirectories containing the tests are named according to the JIRA report describing the problem. The test examples were chosen to demonstrate some of possible cases: they are in different formats (JUnit and pure Java), include heterogeneous sources (Java, C/C++, Java assembler), and some of them are using special launching methods for execution (i.e. custom launchers overwriting default test launching way).
The package for regression tests was chosen as org.apache.harmony.drlvm.tests.regression.h<xxxx>. Where <xxxx> is a number of JIRA report.
Currently the implementation does not support automatic test integration and all of the tests to be executed are explicitly listed in build/make/targets/reg.test.run.xml file. Automatic integration can be easily done after defining of test naming conventions and directory structure.
Please review the proposed infrastructure and provide your comments.
Thanks,
Alexander
> [drlvm][regression] Regression Testing Infrastructure proposal
> --------------------------------------------------------------
>
> Key: HARMONY-2283
> URL: http://issues.apache.org/jira/browse/HARMONY-2283
> Project: Harmony
> Issue Type: Improvement
> Components: DRLVM
> Reporter: Alexander Kleymenov
> Attachments: patch-reg-tests.txt
>
>
> There is no infrastructure for automated regression testing in Harmony DRLVM project.
> This JIRA report is intended to initiate the discussions and works on it.
> With taking into account the specific of the product, the following requirements to infrastructure can be defined:
> 1. It should support tests in different formats (JUnit, pure Java, etc) and implemented in heterogeneous sources (Java, C/C++, Java assembler etc).
> 2. It should provide convenient means for test run configuration (parallel/sequential execution, execution timeouts, command line parameters specifying, environment variables presetting).
> 3. It should provide means for rapid and convenient integration of new test.
> 4. It should be based on commonly used means.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira