You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Andrey Chernyshev <a....@gmail.com> on 2006/06/23 20:00:16 UTC
[DRLVM] Integration checks (was building from svn on FC5)
Folks,
Once we have started to do the changes for drlvm, may be it is a good
time to think about what tests should be passed prior to changes
integration (and what kind of verification to expect before sending a
patch). So far the following comes to my mind:
- running "build test" (it executes a set of smoke tests in drlvm);
- run Eclipse;
- anything else?
Will it make sense to agree on some pre-integration check procedure
for the changes in drlvm? We may probably also need to define the list
of platforms/configurations covered by this procedure. Other approach
could be to work out some regular testing infrastructure which would
monitor the issues, but I'm assuming this won't come quickly.
Thoughts?
Thank you,
Andrey Chernyshev
Intel Middleware Products Division
On 6/23/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> Odd. Nothing should have changed re log4cxx
>
> Sure nothing else changed?
>
> geir
>
> neelakan@uiuc.edu wrote:
> > I just tried building DRLVM out of svn on FC5, and it has a
> > build error that I've seen before. This issue was resolved
> > when I was using r413745, but now I am using r416471 and it
> > is back. The error is below:
> >
> > Thanks,
> > Naveen
> >
> >
> > build.native.cpp:
> > [cc] Starting dependency analysis for 136 files.
> > [cc] 136 files are up to date.
> > [cc] 0 files to be recompiled from dependency
> > analysis.
> > [cc] 5 total files to be compiled.
> >
> > [cc] /work/nneelaka/Sandbox/HarmonyVM/build/lnx_ia32_gcc_rele
> > ase/semis/extra/log4cxx/src/include/log4cxx/xml/domconfigurat
> > or.h:243: error: extra qualification ?
> > log4cxx::xml::DOMConfigurator::? on member ?subst?
> >
> > [cc] /work/nneelaka/Sandbox/HarmonyVM/build/lnx_ia32_gcc_rele
> > ase/semis/extra/log4cxx/src/include/log4cxx/helpers/unicodehe
> > lper.h:98: error: extra qualification ?
> > log4cxx::helpers::UnicodeHelper::? on member ?lengthUTF8?
> >
> > [cc] /work/nneelaka/Sandbox/HarmonyVM/build/lnx_ia32_gcc_rele
> > ase/semis/extra/log4cxx/src/include/log4cxx/helpers/unicodehe
> > lper.h:98: error: extra qualification ?
> > log4cxx::helpers::UnicodeHelper::? on member ?lengthUTF8?
> >
> > [cc] /work/nneelaka/Sandbox/HarmonyVM/build/lnx_ia32_gcc_rele
> > ase/semis/extra/log4cxx/src/include/log4cxx/xml/domconfigurat
> > or.h:243: error: extra qualification ?
> > log4cxx::xml::DOMConfigurator::? on member ?subst?
> >
> > [cc] /work/nneelaka/Sandbox/HarmonyVM/build/lnx_ia32_gcc_rele
> > ase/semis/extra/log4cxx/src/include/log4cxx/helpers/unicodehe
> > lper.h:98: error: extra qualification ?
> > log4cxx::helpers::UnicodeHelper::? on member ?lengthUTF8?
> >
> > On 6/10/06, Mark Hindess <ma...@googlemail.com> wrote:
> >>
> >> On 10 June 2006 at 0:03, Naveen Neelakantam
> > <ne...@uiuc.edu> wrote:
> >>> I just tried building DRLVM out of svn. I am running
> > Fedora Core 5
> >>> with gcc version 4.1.0 20060304 (Red Hat 4.1.0-3).
> >>>
> >>> However, I am getting the build error shown below.
> > Previous posts on
> >>> the dev list led me to believe that this issue had been
> > resolved.
> >>> Did a patch get lost in the transition to svn?
> >> Hi Naveen,
> >>
> >> I'm still testing the latest version of Ivan's patch
> > mentioned in the
> >> message below, so it is not committed in svn yet. (I hope
> > to finish
> >> testing it today or tomorrow.) But I don't see a JIRA
> > with Vladimir's
> >> fixes for FC5 so I don't think this will help you.
> >>
> >> Vladimir, can you create a new JIRA with your additional
> > changes (on top
> >> of Ivan's patch or on drlvm/trunk if I've committed Ivan's
> > patch by the
> >> time you read this.)
> >> Regards,
> >> Mark.
> >>
> >>> Thanks,
> >>> Naveen
> >>>
> >>> On May 19, 2006, at 12:57 AM, Vladimir Gorr wrote:
> >>>
> >>>> Small changes are required to fix the build issue for
> > Fedora 5.
> >>>> I've prepared this patch and I'm ready to put it into
> > JIRA.
> >>>> There are two ways to make this thing:
> >>>> - renew the cumulative patch created by Ivan (see JIRA
> > issue below);
> >>>> - attach these changes as separate patch and adding it
> > to the
> >> HARMONY-443.
> >>>> What way is more preferable?
> >>>>
> >>>> Thanks,
> >>>> Vladimir.
> >>>>
> >>>> On 5/5/06, Ivan Volosyuk <iv...@gmail.com>
> > wrote:
> >>>>> The updated patch "DRLVM-GCC-3.4_and_4.x-
> > cumulative.patch" is now in
> >>>>> Harmony-443 JIRA with detailed instructions.
> >>>>>
> >>>>> http://issues.apache.org/jira/browse/HARMONY-443
> >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Anton Luht <an...@gmail.com>.
Hello,
I've created a patch that adds task 'eclipsehwa' (Eclipse 'Hello,
world!' application) to Harmony classlib build.xml . It's attached to
the issue.
You can run
ant -Dharmony.vm.exe=... -Declipse-home=<eclipse 3.2 root> eclipsehwa
and Eclipse will start, create a project, create a test and compile it
Now it doesn't work with Harmony DRLVM because '-jar' launching is used.
On 7/4/06, Anton Luht <an...@gmail.com> wrote:
> Hello,
>
> I've created an Eclipse automated test based on Salikh's code - please see
>
> http://issues.apache.org/jira/browse/HARMONY-752
>
> I've tested it on Windows XP on Eclipse that goes with harmony VM -
> eclipse-SDK-3.1.1-win32.zip
>
> Sometimes (irregularry) after execution it prints some stack traces to
> the console. Scenario is executed successfully anyway - a project is
> created, editor opens, some code is pasted in it, project is built,
> closed and deleted.
>
> Please feel free to add comments to that issue if you find errors in
> it or fail to run it on some configurations.
> --
> Regards,
> Anton Luht,
> Intel Middleware Products Division
>
--
Regards,
Anton Luht,
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Anton Luht <an...@gmail.com>.
Hello,
I've created an Eclipse automated test based on Salikh's code - please see
http://issues.apache.org/jira/browse/HARMONY-752
I've tested it on Windows XP on Eclipse that goes with harmony VM -
eclipse-SDK-3.1.1-win32.zip
Sometimes (irregularry) after execution it prints some stack traces to
the console. Scenario is executed successfully anyway - a project is
created, editor opens, some code is pasted in it, project is built,
closed and deleted.
Please feel free to add comments to that issue if you find errors in
it or fail to run it on some configurations.
--
Regards,
Anton Luht,
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Salikh Zakirov <Sa...@Intel.com>.
Vladimir Ivanov wrote:
> Do not you think these should be different Eclipse scenarios?:
> 1) simple scenario that run Eclipse, compile small application and runs it
> 2) simple debug scenario
> 3) scenario that runs ant-builder
> 4-N) other scenarios that emulate user work
Doing multiple scenarios indeed takes us closer to a "functional testing"
approach. This is a valid approach, but is indeed something different
from regular development.
I think that just one scenario will be sufficient for integration tests.
> Am I correct that you are going to use record and play tools?
> Or, these will be test(s) written using Eclipse API?
As far as I heard, using "record and play" tools proves unrobust,
as the test depends on the screen resolution, CPU load etc.
Eclipse API based approach seems to be more appropriate.
Below is the text of my experiment of modeling Java development in Eclipse.
The test below does the following
1. creates new project
2. inserts class Hi into the project
3. build the project
4. runs the project and checks the output stream of the process
The thing that I haven't managed to achieve is running from the command line.
-------------------------
import java.util.HashMap;
import junit.framework.TestCase;
import org.eclipse.core.commands.Command;
import org.eclipse.core.commands.ExecutionEvent;
import org.eclipse.core.resources.IncrementalProjectBuilder;
import org.eclipse.core.runtime.CoreException;
import org.eclipse.debug.core.DebugException;
import org.eclipse.debug.core.ILaunch;
import org.eclipse.debug.core.ILaunchManager;
import org.eclipse.debug.core.Launch;
import org.eclipse.debug.core.model.IProcess;
import org.eclipse.jdt.core.IPackageFragment;
import org.eclipse.jdt.core.IType;
import org.eclipse.jdt.launching.IVMInstall;
import org.eclipse.jdt.launching.IVMRunner;
import org.eclipse.jdt.launching.JavaRuntime;
import org.eclipse.jdt.launching.VMRunnerConfiguration;
import org.eclipse.ui.IWorkbench;
import org.eclipse.ui.PlatformUI;
import org.eclipse.ui.commands.ICommandService;
import org.eclipse.ui.console.ConsolePlugin;
public class ClassCreateAndRunTest extends TestCase {
protected void setUp() throws Exception {
}
public void testHi() throws CoreException {
TestProject testProject = new TestProject();
IPackageFragment pack = testProject.createPackage("hi");
IType type = testProject.createType(pack, "Hi.java",
"public class Hi {"
+ " public static void main(String[] args) {"
+ " System.out.println(\"Hi!\");" + " }" + "}");
testProject.getProject().build(
IncrementalProjectBuilder.INCREMENTAL_BUILD, null);
IVMInstall vmInstall = JavaRuntime.getVMInstall(testProject
.getJavaProject());
if (vmInstall == null)
vmInstall = JavaRuntime.getDefaultVMInstall();
if (vmInstall != null) {
IVMRunner vmRunner = vmInstall.getVMRunner(ILaunchManager.RUN_MODE);
if (vmRunner != null) {
String[] classPath = null;
try {
classPath = JavaRuntime
.computeDefaultRuntimeClassPath(testProject
.getJavaProject());
} catch (CoreException e) {
}
if (classPath != null) {
VMRunnerConfiguration vmConfig = new VMRunnerConfiguration(
"hi.Hi", classPath);
vmConfig.setWorkingDirectory("c:\\");
ILaunch launch = new Launch(null, ILaunchManager.RUN_MODE,
null);
vmRunner.run(vmConfig, launch, null);
IProcess[] processes = launch.getProcesses();
assertEquals(1, processes.length);
int timeout = 3000;
while (timeout > 0) {
try {
assertEquals("Non-0 status code", 0, processes[0]
.getExitValue());
break;
} catch (DebugException e) {
timeout -= 100;
try {
Thread.sleep(100);
} catch (InterruptedException ee) {
}
}
}
assertTrue("timed out", timeout > 0);
String out = processes[0].getStreamsProxy()
.getOutputStreamMonitor().getContents();
assertTrue(out.contains("Hi!"));
try {
Thread.sleep(500);
} catch (InterruptedException e) {
}
}
}
}
}
}
import org.eclipse.core.resources.IFolder;
import org.eclipse.core.resources.IProject;
import org.eclipse.core.resources.IProjectDescription;
import org.eclipse.core.resources.IWorkspaceRoot;
import org.eclipse.core.resources.ResourcesPlugin;
import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.IPath;
import org.eclipse.jdt.core.IClasspathEntry;
import org.eclipse.jdt.core.ICompilationUnit;
import org.eclipse.jdt.core.IJavaProject;
import org.eclipse.jdt.core.IPackageFragment;
import org.eclipse.jdt.core.IPackageFragmentRoot;
import org.eclipse.jdt.core.IType;
import org.eclipse.jdt.core.JavaCore;
import org.eclipse.jdt.core.JavaModelException;
import org.eclipse.jdt.launching.JavaRuntime;
class TestProject {
private IProject project;
private IJavaProject javaProject;
private IPackageFragmentRoot sourceFolder;
public TestProject() throws CoreException {
IWorkspaceRoot root = ResourcesPlugin.getWorkspace().getRoot();
project = root.getProject("Eclipse scenario");
project.create(null);
project.open(null);
javaProject = JavaCore.create(project);
IFolder binFolder = createBinFolder();
setJavaNature();
javaProject.setRawClasspath(new IClasspathEntry[0], null);
createOutputFolder(binFolder);
addSystemLibraries();
}
private IFolder createBinFolder() throws CoreException {
IFolder binFolder = project.getFolder("bin");
binFolder.create(false, true, null);
return binFolder;
}
private void setJavaNature() throws CoreException {
IProjectDescription description = project.getDescription();
description.setNatureIds(new String[] { JavaCore.NATURE_ID });
project.setDescription(description, null);
}
private void createOutputFolder(IFolder binFolder)
throws JavaModelException {
IPath outputLocation = binFolder.getFullPath();
javaProject.setOutputLocation(outputLocation, null);
}
public IPackageFragment createPackage(String name) throws CoreException {
if (sourceFolder == null)
sourceFolder = createSourceFolder();
return sourceFolder.createPackageFragment(name, false, null);
}
private IPackageFragmentRoot createSourceFolder() throws CoreException {
IFolder folder = project.getFolder("src");
folder.create(false, true, null);
IPackageFragmentRoot root = javaProject.getPackageFragmentRoot(folder);
IClasspathEntry[] oldEntries = javaProject.getRawClasspath();
IClasspathEntry[] newEntries = new IClasspathEntry[oldEntries.length + 1];
System.arraycopy(oldEntries, 0, newEntries, 0, oldEntries.length);
newEntries[oldEntries.length] = JavaCore.newSourceEntry(root.getPath());
javaProject.setRawClasspath(newEntries, null);
return root;
}
private void addSystemLibraries() throws JavaModelException {
IClasspathEntry[] oldEntries = javaProject.getRawClasspath();
IClasspathEntry[] newEntries = new IClasspathEntry[oldEntries.length + 1];
System.arraycopy(oldEntries, 0, newEntries, 0, oldEntries.length);
newEntries[oldEntries.length] = JavaRuntime
.getDefaultJREContainerEntry();
javaProject.setRawClasspath(newEntries, null);
}
public IType createType(IPackageFragment pack, String cuName, String source)
throws JavaModelException {
StringBuffer buf = new StringBuffer();
buf.append("package " + pack.getElementName() + ";\n");
buf.append("\n");
buf.append(source);
ICompilationUnit cu = pack.createCompilationUnit(cuName,
buf.toString(), false, null);
return cu.getTypes()[0];
}
public IProject getProject() {
return project;
}
public IJavaProject getJavaProject() {
return javaProject;
}
}
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Anton Luht <an...@gmail.com>.
Vladimir,
Regarding what tool will be used - I don't know yet. I'm studying
Eclipse plugins right now. Seems like the situation with standard
macro recording and playback in Eclipse is not very good now [1], so
the emulation will be based on one of Eclipse plugins or will be a
hand-written thing.
I plan to start with 1) and if it succeeds - add other scenarios later.
[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=80140
On 6/27/06, Vladimir Ivanov <iv...@gmail.com> wrote:
> Do not you think these should be different Eclipse scenarios?:
> 1) simple scenario that run Eclipse, compile small application and runs it
> 2) simple debug scenario
> 3) scenario that runs ant-builder
> 4-N) other scenarios that emulate user work
>
> Am I correct that you are going to use record and play tools?
> Or, these will be test(s) written using Eclipse API?
>
> Thanks, Vladimir
>
> On 6/27/06, Anton Luht <an...@gmail.com> wrote:
> >
> > Good day,
> >
> > Anyway it seems that creating an automated Eclipse scenario as
> > described by Salikh will be useful. I'll try to create the scenario,
> > corresponding build target and file a JIRA issue with patch for the
> > new functionality.
> >
> > On 6/27/06, Vladimir Ivanov <iv...@gmail.com> wrote:
> > > Seems, it is not so easy to define the proper test set...
> > > Let's define target to run the integration test. It may be:
> > > 1) we want to be sure that all were integrated correctly?
> > > 2) or we want to guaranty the 'product quality' for build?
> > > 3) some other?
> > >
> > > If target is 1) than we should run minimal tests set (seems, classlib
> > unit
> > > tests over tested VM will enough) on one platform.
> > > If target is 2) than each developer should run all known/defined tests
> > over
> > > all platforms. Seems, is no time for development any more. Everyone will
> > do
> > > the release engineering (RE) work.
> > >
> > > So we have 2 questions here:
> > > 1) the small list of integration test should be defined. It may be
> > subset of
> > > API unit tests collected as 1 or 2 tests from each API area just to be
> > sure
> > > that all things were integrated successfully.
> > > 2) the RE procedure should be defined. Who is responsible to build the
> > HDK
> > > and place it to download page? What tests should be run before it? How
> > often
> > > it should be doing?
> > > It is not so obvious as 1). The procedure may be defined, for example,
> > as:
> > > - one of committers prepare binary form of HDK and test it on one
> > platform.
> > >
> > > - if all tests passed he placed it to download somewhere and
> > > - other people test it on other platform.
> > > - if all tests passed the binaries are promoted and placed to the
> > > 'official' download page.
> > >
> > > Thanks, Vladimir
> > >
> > > PS. To run some scenario tests actually not integration but functional
> > > testing.
> > >
> > > On 6/26/06, Salikh Zakirov < Salikh.Zakirov@intel.com> wrote:
> > > >
> > > > Alexey Petrenko wrote:
> > > > > Some checks before commits are defenetly good...
> > > > >
> > > > > 2006/6/23, Andrey Chernyshev < a.y.chernyshev@gmail.com>:
> > > > >> We may probably also need to define the list of
> > > > >> platforms/configurations covered by this procedure.
> > > > > I'm not sure that I get your idea correctly.
> > > > > Do you suggest to ask every developer to make some checks on
> > different
> > > > > platforms and software configurations?
> > > > > If so... Yes, it is good for product stability.
> > > > > But it will be nearly impossible because very small number of
> > > > > developers have access to different platforms and software
> > > > > configurations...
> > > >
> > > > First and foremost question is *what* to run as integration tests,
> > > > rather than on what platforms. I think we need to define what use
> > cases
> > > > we care for in the form of integration tests.
> > > > The more conveniently the integration tests are packaged, the higher
> > is
> > > > the probability of anyone running them.
> > > > The good example is the "smoke tests" included with DRLVM: they can be
> > > > built and run
> > > > with a single command 'build.bat test' (' build.sh test' on Linux).
> > > >
> > > > Once the integration test set is defined, we can think of platform
> > > > coverage.
> > > > BuildBot [1] could be the way to interested parties to contribute CPU
> > > > cycles
> > > > to verify Harmony quality.
> > > >
> > > > [1] http://buildbot.sourceforge.net/
> > > >
> > > > ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > >
> >
> >
> > --
> > Regards,
> > Anton Luht,
> > Intel Middleware Products Division
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
--
Regards,
Anton Luht,
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Vladimir Ivanov <iv...@gmail.com>.
Do not you think these should be different Eclipse scenarios?:
1) simple scenario that run Eclipse, compile small application and runs it
2) simple debug scenario
3) scenario that runs ant-builder
4-N) other scenarios that emulate user work
Am I correct that you are going to use record and play tools?
Or, these will be test(s) written using Eclipse API?
Thanks, Vladimir
On 6/27/06, Anton Luht <an...@gmail.com> wrote:
>
> Good day,
>
> Anyway it seems that creating an automated Eclipse scenario as
> described by Salikh will be useful. I'll try to create the scenario,
> corresponding build target and file a JIRA issue with patch for the
> new functionality.
>
> On 6/27/06, Vladimir Ivanov <iv...@gmail.com> wrote:
> > Seems, it is not so easy to define the proper test set...
> > Let's define target to run the integration test. It may be:
> > 1) we want to be sure that all were integrated correctly?
> > 2) or we want to guaranty the 'product quality' for build?
> > 3) some other?
> >
> > If target is 1) than we should run minimal tests set (seems, classlib
> unit
> > tests over tested VM will enough) on one platform.
> > If target is 2) than each developer should run all known/defined tests
> over
> > all platforms. Seems, is no time for development any more. Everyone will
> do
> > the release engineering (RE) work.
> >
> > So we have 2 questions here:
> > 1) the small list of integration test should be defined. It may be
> subset of
> > API unit tests collected as 1 or 2 tests from each API area just to be
> sure
> > that all things were integrated successfully.
> > 2) the RE procedure should be defined. Who is responsible to build the
> HDK
> > and place it to download page? What tests should be run before it? How
> often
> > it should be doing?
> > It is not so obvious as 1). The procedure may be defined, for example,
> as:
> > - one of committers prepare binary form of HDK and test it on one
> platform.
> >
> > - if all tests passed he placed it to download somewhere and
> > - other people test it on other platform.
> > - if all tests passed the binaries are promoted and placed to the
> > 'official' download page.
> >
> > Thanks, Vladimir
> >
> > PS. To run some scenario tests actually not integration but functional
> > testing.
> >
> > On 6/26/06, Salikh Zakirov < Salikh.Zakirov@intel.com> wrote:
> > >
> > > Alexey Petrenko wrote:
> > > > Some checks before commits are defenetly good...
> > > >
> > > > 2006/6/23, Andrey Chernyshev < a.y.chernyshev@gmail.com>:
> > > >> We may probably also need to define the list of
> > > >> platforms/configurations covered by this procedure.
> > > > I'm not sure that I get your idea correctly.
> > > > Do you suggest to ask every developer to make some checks on
> different
> > > > platforms and software configurations?
> > > > If so... Yes, it is good for product stability.
> > > > But it will be nearly impossible because very small number of
> > > > developers have access to different platforms and software
> > > > configurations...
> > >
> > > First and foremost question is *what* to run as integration tests,
> > > rather than on what platforms. I think we need to define what use
> cases
> > > we care for in the form of integration tests.
> > > The more conveniently the integration tests are packaged, the higher
> is
> > > the probability of anyone running them.
> > > The good example is the "smoke tests" included with DRLVM: they can be
> > > built and run
> > > with a single command 'build.bat test' (' build.sh test' on Linux).
> > >
> > > Once the integration test set is defined, we can think of platform
> > > coverage.
> > > BuildBot [1] could be the way to interested parties to contribute CPU
> > > cycles
> > > to verify Harmony quality.
> > >
> > > [1] http://buildbot.sourceforge.net/
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
>
>
> --
> Regards,
> Anton Luht,
> Intel Middleware Products Division
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Anton Luht <an...@gmail.com>.
Good day,
Anyway it seems that creating an automated Eclipse scenario as
described by Salikh will be useful. I'll try to create the scenario,
corresponding build target and file a JIRA issue with patch for the
new functionality.
On 6/27/06, Vladimir Ivanov <iv...@gmail.com> wrote:
> Seems, it is not so easy to define the proper test set...
> Let's define target to run the integration test. It may be:
> 1) we want to be sure that all were integrated correctly?
> 2) or we want to guaranty the 'product quality' for build?
> 3) some other?
>
> If target is 1) than we should run minimal tests set (seems, classlib unit
> tests over tested VM will enough) on one platform.
> If target is 2) than each developer should run all known/defined tests over
> all platforms. Seems, is no time for development any more. Everyone will do
> the release engineering (RE) work.
>
> So we have 2 questions here:
> 1) the small list of integration test should be defined. It may be subset of
> API unit tests collected as 1 or 2 tests from each API area just to be sure
> that all things were integrated successfully.
> 2) the RE procedure should be defined. Who is responsible to build the HDK
> and place it to download page? What tests should be run before it? How often
> it should be doing?
> It is not so obvious as 1). The procedure may be defined, for example, as:
> - one of committers prepare binary form of HDK and test it on one platform.
>
> - if all tests passed he placed it to download somewhere and
> - other people test it on other platform.
> - if all tests passed the binaries are promoted and placed to the
> 'official' download page.
>
> Thanks, Vladimir
>
> PS. To run some scenario tests actually not integration but functional
> testing.
>
> On 6/26/06, Salikh Zakirov <Sa...@intel.com> wrote:
> >
> > Alexey Petrenko wrote:
> > > Some checks before commits are defenetly good...
> > >
> > > 2006/6/23, Andrey Chernyshev < a.y.chernyshev@gmail.com>:
> > >> We may probably also need to define the list of
> > >> platforms/configurations covered by this procedure.
> > > I'm not sure that I get your idea correctly.
> > > Do you suggest to ask every developer to make some checks on different
> > > platforms and software configurations?
> > > If so... Yes, it is good for product stability.
> > > But it will be nearly impossible because very small number of
> > > developers have access to different platforms and software
> > > configurations...
> >
> > First and foremost question is *what* to run as integration tests,
> > rather than on what platforms. I think we need to define what use cases
> > we care for in the form of integration tests.
> > The more conveniently the integration tests are packaged, the higher is
> > the probability of anyone running them.
> > The good example is the "smoke tests" included with DRLVM: they can be
> > built and run
> > with a single command 'build.bat test' ('build.sh test' on Linux).
> >
> > Once the integration test set is defined, we can think of platform
> > coverage.
> > BuildBot [1] could be the way to interested parties to contribute CPU
> > cycles
> > to verify Harmony quality.
> >
> > [1] http://buildbot.sourceforge.net/
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
--
Regards,
Anton Luht,
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Vladimir Ivanov <iv...@gmail.com>.
Seems, it is not so easy to define the proper test set...
Let's define target to run the integration test. It may be:
1) we want to be sure that all were integrated correctly?
2) or we want to guaranty the 'product quality' for build?
3) some other?
If target is 1) than we should run minimal tests set (seems, classlib unit
tests over tested VM will enough) on one platform.
If target is 2) than each developer should run all known/defined tests over
all platforms. Seems, is no time for development any more. Everyone will do
the release engineering (RE) work.
So we have 2 questions here:
1) the small list of integration test should be defined. It may be subset of
API unit tests collected as 1 or 2 tests from each API area just to be sure
that all things were integrated successfully.
2) the RE procedure should be defined. Who is responsible to build the HDK
and place it to download page? What tests should be run before it? How often
it should be doing?
It is not so obvious as 1). The procedure may be defined, for example, as:
- one of committers prepare binary form of HDK and test it on one platform.
- if all tests passed he placed it to download somewhere and
- other people test it on other platform.
- if all tests passed the binaries are promoted and placed to the
'official' download page.
Thanks, Vladimir
PS. To run some scenario tests actually not integration but functional
testing.
On 6/26/06, Salikh Zakirov <Sa...@intel.com> wrote:
>
> Alexey Petrenko wrote:
> > Some checks before commits are defenetly good...
> >
> > 2006/6/23, Andrey Chernyshev < a.y.chernyshev@gmail.com>:
> >> We may probably also need to define the list of
> >> platforms/configurations covered by this procedure.
> > I'm not sure that I get your idea correctly.
> > Do you suggest to ask every developer to make some checks on different
> > platforms and software configurations?
> > If so... Yes, it is good for product stability.
> > But it will be nearly impossible because very small number of
> > developers have access to different platforms and software
> > configurations...
>
> First and foremost question is *what* to run as integration tests,
> rather than on what platforms. I think we need to define what use cases
> we care for in the form of integration tests.
> The more conveniently the integration tests are packaged, the higher is
> the probability of anyone running them.
> The good example is the "smoke tests" included with DRLVM: they can be
> built and run
> with a single command 'build.bat test' ('build.sh test' on Linux).
>
> Once the integration test set is defined, we can think of platform
> coverage.
> BuildBot [1] could be the way to interested parties to contribute CPU
> cycles
> to verify Harmony quality.
>
> [1] http://buildbot.sourceforge.net/
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Salikh Zakirov <Sa...@Intel.com>.
Alexey Petrenko wrote:
> Some checks before commits are defenetly good...
>
> 2006/6/23, Andrey Chernyshev <a....@gmail.com>:
>> We may probably also need to define the list of
>> platforms/configurations covered by this procedure.
> I'm not sure that I get your idea correctly.
> Do you suggest to ask every developer to make some checks on different
> platforms and software configurations?
> If so... Yes, it is good for product stability.
> But it will be nearly impossible because very small number of
> developers have access to different platforms and software
> configurations...
First and foremost question is *what* to run as integration tests,
rather than on what platforms. I think we need to define what use cases
we care for in the form of integration tests.
The more conveniently the integration tests are packaged, the higher is the probability of anyone running them.
The good example is the "smoke tests" included with DRLVM: they can be built and run
with a single command 'build.bat test' ('build.sh test' on Linux).
Once the integration test set is defined, we can think of platform coverage.
BuildBot [1] could be the way to interested parties to contribute CPU cycles
to verify Harmony quality.
[1] http://buildbot.sourceforge.net/
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Salikh Zakirov <Sa...@Intel.com>.
Anton Luht wrote:
> As far as I understand Andrey, he suggests to run tests when a piece
> of code is integrated into the code base, by the person who has access
> to the repository (committer). A developer shouldn't test his piece of
> code on any platform, though it'd be good if he did it and reported
> results in the JIRA issue.
>
> I don't know if running tests for hours is suitable for integration
> check, if so, I can suggest using Eclipse automated tests (can be
> found in download section for each Eclipse release). Total amount of
> tests is > 10000 . They test Eclipse functionality quite deeply. Note:
> they'll fail without GUI.
IMHO, For integration testing it would be enough to run just one test
that verify that the change is not breaking major use case.
Running Eclipse for Java development is one of the major targets of DRLVM development,
so it would be very convenient to have just *one* Eclipse unit test,
that would model basic Eclipse workflow:
* create project
* inject java code
* compile java code
* (maybe) debug java code: set breakpoints, run the program, inquire variable state, etc.
To make this usable for DRLVM committers, the unit test needs to be readily
available, for example, .jar file committed directly to depends/ or downloadable
from some external source. Its download can be handled in exactly the same
way as other dependencies are downloaded by 'ant fetch-depends'.
As a final result, the committer will be able to run a *single* command,
that will verify that changes in the DRLVM do not break Eclipse use case.
Does this make sense?
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Tim Ellison <t....@gmail.com>.
Alexey Petrenko wrote:
> Yes, I understand this.
>
> But are you sure that every committer now and in the future has access
> to the different platforms and configurations?
>
> Committers, are you? :)
If I'm patching some general Java code then I just test on my local
workstation and wait for the build system to try it elsewhere; but if I
am writing code that is platform-specific then I will test locally on
windows and linux before committing (because I can). As the number of
platforms increases then I'll need access to more architectures to test.
Regards,
Tim
> 2006/6/26, Anton Luht <an...@gmail.com>:
>> Alexey,
>>
>> As far as I understand Andrey, he suggests to run tests when a piece
>> of code is integrated into the code base, by the person who has access
>> to the repository (committer). A developer shouldn't test his piece of
>> code on any platform, though it'd be good if he did it and reported
>> results in the JIRA issue.
>>
>> I don't know if running tests for hours is suitable for integration
>> check, if so, I can suggest using Eclipse automated tests (can be
>> found in download section for each Eclipse release). Total amount of
>> tests is > 10000 . They test Eclipse functionality quite deeply. Note:
>> they'll fail without GUI.
>>
>> > > We may probably also need to define the list of
>> platforms/configurations covered by this procedure.
>> > I'm not sure that I get your idea correctly.
>> > Do you suggest to ask every developer to make some checks on different
>> > platforms and software configurations?
>> > If so... Yes, it is good for product stability.
>> > But it will be nearly impossible because very small number of
>> > developers have access to different platforms and software
>> > configurations...
>>
>> --
>> Regards,
>> Anton Luht,
>> Intel Middleware Products Division
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
>
--
Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Alexey Petrenko <al...@gmail.com>.
Yes, I understand this.
But are you sure that every committer now and in the future has access
to the different platforms and configurations?
Committers, are you? :)
2006/6/26, Anton Luht <an...@gmail.com>:
> Alexey,
>
> As far as I understand Andrey, he suggests to run tests when a piece
> of code is integrated into the code base, by the person who has access
> to the repository (committer). A developer shouldn't test his piece of
> code on any platform, though it'd be good if he did it and reported
> results in the JIRA issue.
>
> I don't know if running tests for hours is suitable for integration
> check, if so, I can suggest using Eclipse automated tests (can be
> found in download section for each Eclipse release). Total amount of
> tests is > 10000 . They test Eclipse functionality quite deeply. Note:
> they'll fail without GUI.
>
> > > We may probably also need to define the list of platforms/configurations covered by this procedure.
> > I'm not sure that I get your idea correctly.
> > Do you suggest to ask every developer to make some checks on different
> > platforms and software configurations?
> > If so... Yes, it is good for product stability.
> > But it will be nearly impossible because very small number of
> > developers have access to different platforms and software
> > configurations...
>
> --
> Regards,
> Anton Luht,
> Intel Middleware Products Division
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Alexey A. Petrenko
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Anton Luht <an...@gmail.com>.
Alexey,
As far as I understand Andrey, he suggests to run tests when a piece
of code is integrated into the code base, by the person who has access
to the repository (committer). A developer shouldn't test his piece of
code on any platform, though it'd be good if he did it and reported
results in the JIRA issue.
I don't know if running tests for hours is suitable for integration
check, if so, I can suggest using Eclipse automated tests (can be
found in download section for each Eclipse release). Total amount of
tests is > 10000 . They test Eclipse functionality quite deeply. Note:
they'll fail without GUI.
> > We may probably also need to define the list of platforms/configurations covered by this procedure.
> I'm not sure that I get your idea correctly.
> Do you suggest to ask every developer to make some checks on different
> platforms and software configurations?
> If so... Yes, it is good for product stability.
> But it will be nearly impossible because very small number of
> developers have access to different platforms and software
> configurations...
--
Regards,
Anton Luht,
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [DRLVM] Integration checks (was building from svn on FC5)
Posted by Alexey Petrenko <al...@gmail.com>.
Some checks before commits are defenetly good...
2006/6/23, Andrey Chernyshev <a....@gmail.com>:
> We may probably also need to define the list of platforms/configurations covered by this procedure.
I'm not sure that I get your idea correctly.
Do you suggest to ask every developer to make some checks on different
platforms and software configurations?
If so... Yes, it is good for product stability.
But it will be nearly impossible because very small number of
developers have access to different platforms and software
configurations...
--
Alexey A. Petrenko
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org