You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Leo Li <li...@gmail.com> on 2006/11/27 04:03:24 UTC
[general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Hi, all:
During fixing the bug of Harmony-2249, I found that the testcase in one
junit test file might lead to other fail in a different junit file. After
digging into it, I am aware that testcase can influence the global state of
a VM, for example, the resolution of class (both RI and Harmony have similar
behavior). Although I changed the testcase as a workaround, it is not
tested so thoroughly as I expected in order not to lead other tests to fail.
So I recommend if we can specify some testcase that should run in junit
with fork = "true". Maybe to add some annotation is one resolution or to add
them to a property file as we done with excluded ones.
Any suggestion?
--
Leo Li
China Software Development Lab, IBM
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Alexei Fedotov <al...@gmail.com>.
Leo,
If I understood you correctly you suggested running all test cases in
separate JVMs. Yes, I agree that this would improve execution
statistics. This can be done right now using a special property
${hy.test.forkmode} added by Vladimir and Mikhail.
>From the other side a user who is running a real application cannot be
advised to launch few next methods in the fresh JVM. That is why JVM,
especially a server JVM, should handle long execution without changing
its global state to inconsistent one.
I personally prefer to add a proper cleanup for all tests and maintain
their ability to run in a single JVM in a way a conventional JUnit
semantics assumes. This would serve as a simplest reliability load for
Harmony, and we, executing tests each day while testing our patches
will maintain reliability level of Harmony on some level. Running
tests in one JVM also gives us a small but nice benefit of speeding up
the test execution.
I could imagine some negative JNI tests which can be impossible to
recover from, and my suggestion does not apply to such tests.
--
Thank you,
Alexei
On 11/27/06, Leo Li <li...@gmail.com> wrote:
> Yes, in some special cases. Both RI and Harmony adopt some cache mechanism
> in resolving class in ObjectInputStream.
>
> On 11/27/06, Geir Magnusson Jr. <ge...@pobox.com> wrote:
> >
> > Right - I guess the ambiguity for me is if the side effect was correct
> > behavior or not - we're using this "run all in one VM" mode in DRLVM
> > tests to find harmful side effects...
> >
> > geir
> >
> > Mikhail Loenko wrote:
> > > When you changed the test case to avoid side-effect have you made the
> > test
> > > weaker? If not I'd prefer to run them in a fork mode "once" because in
> > this
> > > case they run faster.
> > >
> > > Thanks,
> > > Mikhail
> > >
> > > 2006/11/27, Leo Li <li...@gmail.com>:
> > >> Hi, all:
> > >> During fixing the bug of Harmony-2249, I found that the testcase
> > >> in one
> > >> junit test file might lead to other fail in a different junit file.
> > After
> > >> digging into it, I am aware that testcase can influence the global
> > >> state of
> > >> a VM, for example, the resolution of class (both RI and Harmony have
> > >> similar
> > >> behavior). Although I changed the testcase as a workaround, it is not
> > >> tested so thoroughly as I expected in order not to lead other tests to
> > >> fail.
> > >>
> > >> So I recommend if we can specify some testcase that should run in
> > >> junit
> > >> with fork = "true". Maybe to add some annotation is one resolution or
> > >> to add
> > >> them to a property file as we done with excluded ones.
> > >> Any suggestion?
> > >>
> > >> --
> > >> Leo Li
> > >> China Software Development Lab, IBM
> > >>
> > >>
> >
>
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Leo Li <li...@gmail.com>.
Yes, in some special cases. Both RI and Harmony adopt some cache mechanism
in resolving class in ObjectInputStream.
On 11/27/06, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>
> Right - I guess the ambiguity for me is if the side effect was correct
> behavior or not - we're using this "run all in one VM" mode in DRLVM
> tests to find harmful side effects...
>
> geir
>
> Mikhail Loenko wrote:
> > When you changed the test case to avoid side-effect have you made the
> test
> > weaker? If not I'd prefer to run them in a fork mode "once" because in
> this
> > case they run faster.
> >
> > Thanks,
> > Mikhail
> >
> > 2006/11/27, Leo Li <li...@gmail.com>:
> >> Hi, all:
> >> During fixing the bug of Harmony-2249, I found that the testcase
> >> in one
> >> junit test file might lead to other fail in a different junit file.
> After
> >> digging into it, I am aware that testcase can influence the global
> >> state of
> >> a VM, for example, the resolution of class (both RI and Harmony have
> >> similar
> >> behavior). Although I changed the testcase as a workaround, it is not
> >> tested so thoroughly as I expected in order not to lead other tests to
> >> fail.
> >>
> >> So I recommend if we can specify some testcase that should run in
> >> junit
> >> with fork = "true". Maybe to add some annotation is one resolution or
> >> to add
> >> them to a property file as we done with excluded ones.
> >> Any suggestion?
> >>
> >> --
> >> Leo Li
> >> China Software Development Lab, IBM
> >>
> >>
>
--
Leo Li
China Software Development Lab, IBM
Re: [general][tests]To restrict the side-effect of junit tests by
launching it in a different VM.
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
Right - I guess the ambiguity for me is if the side effect was correct
behavior or not - we're using this "run all in one VM" mode in DRLVM
tests to find harmful side effects...
geir
Mikhail Loenko wrote:
> When you changed the test case to avoid side-effect have you made the test
> weaker? If not I'd prefer to run them in a fork mode "once" because in this
> case they run faster.
>
> Thanks,
> Mikhail
>
> 2006/11/27, Leo Li <li...@gmail.com>:
>> Hi, all:
>> During fixing the bug of Harmony-2249, I found that the testcase
>> in one
>> junit test file might lead to other fail in a different junit file. After
>> digging into it, I am aware that testcase can influence the global
>> state of
>> a VM, for example, the resolution of class (both RI and Harmony have
>> similar
>> behavior). Although I changed the testcase as a workaround, it is not
>> tested so thoroughly as I expected in order not to lead other tests to
>> fail.
>>
>> So I recommend if we can specify some testcase that should run in
>> junit
>> with fork = "true". Maybe to add some annotation is one resolution or
>> to add
>> them to a property file as we done with excluded ones.
>> Any suggestion?
>>
>> --
>> Leo Li
>> China Software Development Lab, IBM
>>
>>
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Leo Li <li...@gmail.com>.
Yes, to some degree. Since some special class, such as the primitive types,
long.class can not be substituted.
On 11/27/06, Mikhail Loenko <ml...@gmail.com> wrote:
>
> When you changed the test case to avoid side-effect have you made the test
> weaker? If not I'd prefer to run them in a fork mode "once" because in
> this
> case they run faster.
>
> Thanks,
> Mikhail
>
> 2006/11/27, Leo Li <li...@gmail.com>:
> > Hi, all:
> > During fixing the bug of Harmony-2249, I found that the testcase in
> one
> > junit test file might lead to other fail in a different junit file.
> After
> > digging into it, I am aware that testcase can influence the global state
> of
> > a VM, for example, the resolution of class (both RI and Harmony have
> similar
> > behavior). Although I changed the testcase as a workaround, it is not
> > tested so thoroughly as I expected in order not to lead other tests to
> fail.
> >
> > So I recommend if we can specify some testcase that should run in
> junit
> > with fork = "true". Maybe to add some annotation is one resolution or to
> add
> > them to a property file as we done with excluded ones.
> > Any suggestion?
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> >
> >
>
--
Leo Li
China Software Development Lab, IBM
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Mikhail Loenko <ml...@gmail.com>.
When you changed the test case to avoid side-effect have you made the test
weaker? If not I'd prefer to run them in a fork mode "once" because in this
case they run faster.
Thanks,
Mikhail
2006/11/27, Leo Li <li...@gmail.com>:
> Hi, all:
> During fixing the bug of Harmony-2249, I found that the testcase in one
> junit test file might lead to other fail in a different junit file. After
> digging into it, I am aware that testcase can influence the global state of
> a VM, for example, the resolution of class (both RI and Harmony have similar
> behavior). Although I changed the testcase as a workaround, it is not
> tested so thoroughly as I expected in order not to lead other tests to fail.
>
> So I recommend if we can specify some testcase that should run in junit
> with fork = "true". Maybe to add some annotation is one resolution or to add
> them to a property file as we done with excluded ones.
> Any suggestion?
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Leo Li <li...@gmail.com>.
Yes. Exec really solves the problem.
On 12/8/06, Alexei Fedotov <al...@gmail.com> wrote:
>
> Please, don't mind: compared to this solution it's better to use exec().
>
> On 12/8/06, Alexei Fedotov <al...@gmail.com> wrote:
> > Leo,
> > Thank you for explaining! I wonder if you read enough correct objects,
> > would it clear the cache from incorrect ones?
> >
> [...]
>
--
Leo Li
China Software Development Lab, IBM
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Alexei Fedotov <al...@gmail.com>.
Please, don't mind: compared to this solution it's better to use exec().
On 12/8/06, Alexei Fedotov <al...@gmail.com> wrote:
> Leo,
> Thank you for explaining! I wonder if you read enough correct objects,
> would it clear the cache from incorrect ones?
>
[...]
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Alexei Fedotov <al...@gmail.com>.
Leo,
Thank you for explaining! I wonder if you read enough correct objects,
would it clear the cache from incorrect ones?
On 11/29/06, Leo Li <li...@gmail.com> wrote:
> Hi, Alexei:
> Excuse me if I confusing you. Actually it is in the original patch. It
> is about the caching mechanism adopted by ObjectInputStream in resolving
> class.
> Once I have written such a testcase:
>
> public static class ObjectOutputStreamWithWriteDesc extends
> ObjectOutputStream {
> public ObjectOutputStreamWithWriteDesc(OutputStream os)
> throws IOException {
> super(os);
> }
>
> public void writeClassDescriptor(ObjectStreamClass desc)
> throws IOException {
> }
> }
>
> public static class ObjectIutputStreamWithReadDesc extends
> ObjectInputStream {
> private Class returnClass;
>
> public ObjectIutputStreamWithReadDesc(InputStream is, Class
> returnClass)
> throws IOException {
> super(is);
> this.returnClass = returnClass;
> }
>
> public ObjectStreamClass readClassDescriptor() throws IOException,
> ClassNotFoundException {
> return ObjectStreamClass.lookup(returnClass);
> }
> }
>
> public void test_ClassDescriptor() throws IOException,
> ClassNotFoundException {
>
> // Conversion between classes
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> ObjectOutputStreamWithWriteDesc oos = new
> ObjectOutputStreamWithWriteDesc(baos);
> oos.writeObject(String.class);
> oos.close();
>
> Class cls = Integer.class;
> bais = new ByteArrayInputStream(baos.toByteArray());
> ObjectIutputStreamWithReadDesc ois = new
> ObjectIutputStreamWithReadDesc(bais, cls);
> * //Here I changed the information of Integer Class Desc in
> ObjectInputStream's cache not so as normal.
> * Object obj = ois.readObject();
> ...}
>
>
> But in SerialStressTest,
> ...
>
> oos.writeObject(new Object[] { Integer.class, new Integer(1) });
> oos.close();
>
> ois = new ObjectInputStreamSubclass(loadStream());
> ois.readObject();
> ois.close();
>
> Class[] resolvedClasses = ((ObjectInputStreamSubclass) ois)
> .resolvedClasses();
>
> but here is resolvedClasses there are only two class: Object.class and
> Integer.class, but without Number.class which is expected in the normal
> case. This is because I inited the ObjectInputStream' s caching state of
> Integer.class unnormally in the first testcase. Furthermore, through public
> interface, I cannot rollback the state.(RI will pass this example, but it
> also has its own caching mechanism so I can site another problem to let it
> behave different from normal.)
>
> On 11/29/06, Alexei Fedotov <al...@gmail.com> wrote:
> >
> > Leo,
> > Could you please share more info about the problem?
> >
> > Could you please name a test case which affects other test cases? I
> > have tried to check http://issues.apache.org/jira/browse/HARMONY-2249
> > but it doesn't contain any clue.
> >
> > Why this test case cannot be fixed to clean environment on exit?
> > --
> > Thank you,
> > Alexei
> >
> >
> > On 11/29/06, Leo Li <li...@gmail.com> wrote:
> > > Anyway, using exec, the test is running in a standalone java program
> > rather
> > > than the normal junit test.:)
> > >
> > > On 11/28/06, Stepan Mishura <st...@gmail.com> wrote:
> > > >
> > > > On 11/28/06, Leo Li wrote:
> > > > >
> > > > > OK. It will do in exec, but the style is a little different.:)
> > > >
> > > >
> > > > Sorry, I didn't catch - what "different style" means here?
> > > >
> > > > Thanks,
> > > > Stepan.
> > > >
> > > > And I also believe run most tests in one VM will save time.(Actually
> > it
> > > > > has been quite long currently.)
> > > > > I just want to denote the tests that should run in seperate VM while
> > > > > remaining the style of junit tests except some configurations. (Like
> > > > > something in AOP and without intruding.)
> > > > >
> > > > >
> > > > > On 11/28/06, Tim Ellison <t....@gmail.com> wrote:
> > > > > >
> > > > > > Stepan Mishura wrote:
> > > > > > > On 11/27/06, Leo Li wrote:
> > > > > > >>
> > > > > > >> Hi, all:
> > > > > > >> During fixing the bug of Harmony-2249, I found that the
> > > > testcase
> > > > > in
> > > > > > >> one
> > > > > > >> junit test file might lead to other fail in a different junit
> > file.
> > > > > > After
> > > > > > >> digging into it, I am aware that testcase can influence the
> > global
> > > > > > state
> > > > > > >> of
> > > > > > >> a VM, for example, the resolution of class (both RI and Harmony
> > > > have
> > > > > > >> similar
> > > > > > >> behavior). Although I changed the testcase as a workaround, it
> > is
> > > > > not
> > > > > > >> tested so thoroughly as I expected in order not to lead other
> > tests
> > > > > to
> > > > > > >> fail.
> > > > > > >
> > > > > > >
> > > > > > > If a test's execution influence of VM state and this is critical
> > for
> > > > > > other
> > > > > > > test then the test can fork VM (via Support_Exec.execJava()) and
> > do
> > > > > all
> > > > > > > testing in the forked VM.
> > > > > >
> > > > > > +1 -- and this should be the exception, in general tests should
> > put
> > > > > > things back as they found them. exec'ing a new Java is for those
> > > > cases
> > > > > > where you cannot do that.
> > > > > >
> > > > > > Regards,
> > > > > > Tim
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Tim Ellison (t.p.ellison@gmail.com)
> > > > > > IBM Java technology centre, UK.
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Leo Li
> > > > > China Software Development Lab, IBM
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Stepan Mishura
> > > > Intel Middleware Products Division
> > > >
> > > >
> > >
> > >
> > > --
> > > Leo Li
> > > China Software Development Lab, IBM
> > >
> > >
> >
> >
> > --
> > Thank you,
> > Alexei
> >
>
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>
--
With best regards,
Alexei,
ESSD, Intel
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Leo Li <li...@gmail.com>.
Hi, Alexei:
Excuse me if I confusing you. Actually it is in the original patch. It
is about the caching mechanism adopted by ObjectInputStream in resolving
class.
Once I have written such a testcase:
public static class ObjectOutputStreamWithWriteDesc extends
ObjectOutputStream {
public ObjectOutputStreamWithWriteDesc(OutputStream os)
throws IOException {
super(os);
}
public void writeClassDescriptor(ObjectStreamClass desc)
throws IOException {
}
}
public static class ObjectIutputStreamWithReadDesc extends
ObjectInputStream {
private Class returnClass;
public ObjectIutputStreamWithReadDesc(InputStream is, Class
returnClass)
throws IOException {
super(is);
this.returnClass = returnClass;
}
public ObjectStreamClass readClassDescriptor() throws IOException,
ClassNotFoundException {
return ObjectStreamClass.lookup(returnClass);
}
}
public void test_ClassDescriptor() throws IOException,
ClassNotFoundException {
// Conversion between classes
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStreamWithWriteDesc oos = new
ObjectOutputStreamWithWriteDesc(baos);
oos.writeObject(String.class);
oos.close();
Class cls = Integer.class;
bais = new ByteArrayInputStream(baos.toByteArray());
ObjectIutputStreamWithReadDesc ois = new
ObjectIutputStreamWithReadDesc(bais, cls);
* //Here I changed the information of Integer Class Desc in
ObjectInputStream's cache not so as normal.
* Object obj = ois.readObject();
...}
But in SerialStressTest,
...
oos.writeObject(new Object[] { Integer.class, new Integer(1) });
oos.close();
ois = new ObjectInputStreamSubclass(loadStream());
ois.readObject();
ois.close();
Class[] resolvedClasses = ((ObjectInputStreamSubclass) ois)
.resolvedClasses();
but here is resolvedClasses there are only two class: Object.class and
Integer.class, but without Number.class which is expected in the normal
case. This is because I inited the ObjectInputStream' s caching state of
Integer.class unnormally in the first testcase. Furthermore, through public
interface, I cannot rollback the state.(RI will pass this example, but it
also has its own caching mechanism so I can site another problem to let it
behave different from normal.)
On 11/29/06, Alexei Fedotov <al...@gmail.com> wrote:
>
> Leo,
> Could you please share more info about the problem?
>
> Could you please name a test case which affects other test cases? I
> have tried to check http://issues.apache.org/jira/browse/HARMONY-2249
> but it doesn't contain any clue.
>
> Why this test case cannot be fixed to clean environment on exit?
> --
> Thank you,
> Alexei
>
>
> On 11/29/06, Leo Li <li...@gmail.com> wrote:
> > Anyway, using exec, the test is running in a standalone java program
> rather
> > than the normal junit test.:)
> >
> > On 11/28/06, Stepan Mishura <st...@gmail.com> wrote:
> > >
> > > On 11/28/06, Leo Li wrote:
> > > >
> > > > OK. It will do in exec, but the style is a little different.:)
> > >
> > >
> > > Sorry, I didn't catch - what "different style" means here?
> > >
> > > Thanks,
> > > Stepan.
> > >
> > > And I also believe run most tests in one VM will save time.(Actually
> it
> > > > has been quite long currently.)
> > > > I just want to denote the tests that should run in seperate VM while
> > > > remaining the style of junit tests except some configurations. (Like
> > > > something in AOP and without intruding.)
> > > >
> > > >
> > > > On 11/28/06, Tim Ellison <t....@gmail.com> wrote:
> > > > >
> > > > > Stepan Mishura wrote:
> > > > > > On 11/27/06, Leo Li wrote:
> > > > > >>
> > > > > >> Hi, all:
> > > > > >> During fixing the bug of Harmony-2249, I found that the
> > > testcase
> > > > in
> > > > > >> one
> > > > > >> junit test file might lead to other fail in a different junit
> file.
> > > > > After
> > > > > >> digging into it, I am aware that testcase can influence the
> global
> > > > > state
> > > > > >> of
> > > > > >> a VM, for example, the resolution of class (both RI and Harmony
> > > have
> > > > > >> similar
> > > > > >> behavior). Although I changed the testcase as a workaround, it
> is
> > > > not
> > > > > >> tested so thoroughly as I expected in order not to lead other
> tests
> > > > to
> > > > > >> fail.
> > > > > >
> > > > > >
> > > > > > If a test's execution influence of VM state and this is critical
> for
> > > > > other
> > > > > > test then the test can fork VM (via Support_Exec.execJava()) and
> do
> > > > all
> > > > > > testing in the forked VM.
> > > > >
> > > > > +1 -- and this should be the exception, in general tests should
> put
> > > > > things back as they found them. exec'ing a new Java is for those
> > > cases
> > > > > where you cannot do that.
> > > > >
> > > > > Regards,
> > > > > Tim
> > > > >
> > > > > --
> > > > >
> > > > > Tim Ellison (t.p.ellison@gmail.com)
> > > > > IBM Java technology centre, UK.
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Leo Li
> > > > China Software Development Lab, IBM
> > > >
> > > >
> > >
> > >
> > > --
> > > Stepan Mishura
> > > Intel Middleware Products Division
> > >
> > >
> >
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> >
> >
>
>
> --
> Thank you,
> Alexei
>
--
Leo Li
China Software Development Lab, IBM
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Alexei Fedotov <al...@gmail.com>.
Leo,
Could you please share more info about the problem?
Could you please name a test case which affects other test cases? I
have tried to check http://issues.apache.org/jira/browse/HARMONY-2249
but it doesn't contain any clue.
Why this test case cannot be fixed to clean environment on exit?
--
Thank you,
Alexei
On 11/29/06, Leo Li <li...@gmail.com> wrote:
> Anyway, using exec, the test is running in a standalone java program rather
> than the normal junit test.:)
>
> On 11/28/06, Stepan Mishura <st...@gmail.com> wrote:
> >
> > On 11/28/06, Leo Li wrote:
> > >
> > > OK. It will do in exec, but the style is a little different.:)
> >
> >
> > Sorry, I didn't catch - what "different style" means here?
> >
> > Thanks,
> > Stepan.
> >
> > And I also believe run most tests in one VM will save time.(Actually it
> > > has been quite long currently.)
> > > I just want to denote the tests that should run in seperate VM while
> > > remaining the style of junit tests except some configurations. (Like
> > > something in AOP and without intruding.)
> > >
> > >
> > > On 11/28/06, Tim Ellison <t....@gmail.com> wrote:
> > > >
> > > > Stepan Mishura wrote:
> > > > > On 11/27/06, Leo Li wrote:
> > > > >>
> > > > >> Hi, all:
> > > > >> During fixing the bug of Harmony-2249, I found that the
> > testcase
> > > in
> > > > >> one
> > > > >> junit test file might lead to other fail in a different junit file.
> > > > After
> > > > >> digging into it, I am aware that testcase can influence the global
> > > > state
> > > > >> of
> > > > >> a VM, for example, the resolution of class (both RI and Harmony
> > have
> > > > >> similar
> > > > >> behavior). Although I changed the testcase as a workaround, it is
> > > not
> > > > >> tested so thoroughly as I expected in order not to lead other tests
> > > to
> > > > >> fail.
> > > > >
> > > > >
> > > > > If a test's execution influence of VM state and this is critical for
> > > > other
> > > > > test then the test can fork VM (via Support_Exec.execJava()) and do
> > > all
> > > > > testing in the forked VM.
> > > >
> > > > +1 -- and this should be the exception, in general tests should put
> > > > things back as they found them. exec'ing a new Java is for those
> > cases
> > > > where you cannot do that.
> > > >
> > > > Regards,
> > > > Tim
> > > >
> > > > --
> > > >
> > > > Tim Ellison (t.p.ellison@gmail.com)
> > > > IBM Java technology centre, UK.
> > > >
> > >
> > >
> > >
> > > --
> > > Leo Li
> > > China Software Development Lab, IBM
> > >
> > >
> >
> >
> > --
> > Stepan Mishura
> > Intel Middleware Products Division
> >
> >
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>
--
Thank you,
Alexei
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Leo Li <li...@gmail.com>.
Anyway, using exec, the test is running in a standalone java program rather
than the normal junit test.:)
On 11/28/06, Stepan Mishura <st...@gmail.com> wrote:
>
> On 11/28/06, Leo Li wrote:
> >
> > OK. It will do in exec, but the style is a little different.:)
>
>
> Sorry, I didn't catch - what "different style" means here?
>
> Thanks,
> Stepan.
>
> And I also believe run most tests in one VM will save time.(Actually it
> > has been quite long currently.)
> > I just want to denote the tests that should run in seperate VM while
> > remaining the style of junit tests except some configurations. (Like
> > something in AOP and without intruding.)
> >
> >
> > On 11/28/06, Tim Ellison <t....@gmail.com> wrote:
> > >
> > > Stepan Mishura wrote:
> > > > On 11/27/06, Leo Li wrote:
> > > >>
> > > >> Hi, all:
> > > >> During fixing the bug of Harmony-2249, I found that the
> testcase
> > in
> > > >> one
> > > >> junit test file might lead to other fail in a different junit file.
> > > After
> > > >> digging into it, I am aware that testcase can influence the global
> > > state
> > > >> of
> > > >> a VM, for example, the resolution of class (both RI and Harmony
> have
> > > >> similar
> > > >> behavior). Although I changed the testcase as a workaround, it is
> > not
> > > >> tested so thoroughly as I expected in order not to lead other tests
> > to
> > > >> fail.
> > > >
> > > >
> > > > If a test's execution influence of VM state and this is critical for
> > > other
> > > > test then the test can fork VM (via Support_Exec.execJava()) and do
> > all
> > > > testing in the forked VM.
> > >
> > > +1 -- and this should be the exception, in general tests should put
> > > things back as they found them. exec'ing a new Java is for those
> cases
> > > where you cannot do that.
> > >
> > > Regards,
> > > Tim
> > >
> > > --
> > >
> > > Tim Ellison (t.p.ellison@gmail.com)
> > > IBM Java technology centre, UK.
> > >
> >
> >
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> >
> >
>
>
> --
> Stepan Mishura
> Intel Middleware Products Division
>
>
--
Leo Li
China Software Development Lab, IBM
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Stepan Mishura <st...@gmail.com>.
On 11/28/06, Leo Li wrote:
>
> OK. It will do in exec, but the style is a little different.:)
Sorry, I didn't catch - what "different style" means here?
Thanks,
Stepan.
And I also believe run most tests in one VM will save time.(Actually it
> has been quite long currently.)
> I just want to denote the tests that should run in seperate VM while
> remaining the style of junit tests except some configurations. (Like
> something in AOP and without intruding.)
>
>
> On 11/28/06, Tim Ellison <t....@gmail.com> wrote:
> >
> > Stepan Mishura wrote:
> > > On 11/27/06, Leo Li wrote:
> > >>
> > >> Hi, all:
> > >> During fixing the bug of Harmony-2249, I found that the testcase
> in
> > >> one
> > >> junit test file might lead to other fail in a different junit file.
> > After
> > >> digging into it, I am aware that testcase can influence the global
> > state
> > >> of
> > >> a VM, for example, the resolution of class (both RI and Harmony have
> > >> similar
> > >> behavior). Although I changed the testcase as a workaround, it is
> not
> > >> tested so thoroughly as I expected in order not to lead other tests
> to
> > >> fail.
> > >
> > >
> > > If a test's execution influence of VM state and this is critical for
> > other
> > > test then the test can fork VM (via Support_Exec.execJava()) and do
> all
> > > testing in the forked VM.
> >
> > +1 -- and this should be the exception, in general tests should put
> > things back as they found them. exec'ing a new Java is for those cases
> > where you cannot do that.
> >
> > Regards,
> > Tim
> >
> > --
> >
> > Tim Ellison (t.p.ellison@gmail.com)
> > IBM Java technology centre, UK.
> >
>
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>
--
Stepan Mishura
Intel Middleware Products Division
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Leo Li <li...@gmail.com>.
OK. It will do in exec, but the style is a little different.:)
And I also believe run most tests in one VM will save time.(Actually it
has been quite long currently.)
I just want to denote the tests that should run in seperate VM while
remaining the style of junit tests except some configurations. (Like
something in AOP and without intruding.)
On 11/28/06, Tim Ellison <t....@gmail.com> wrote:
>
> Stepan Mishura wrote:
> > On 11/27/06, Leo Li wrote:
> >>
> >> Hi, all:
> >> During fixing the bug of Harmony-2249, I found that the testcase in
> >> one
> >> junit test file might lead to other fail in a different junit file.
> After
> >> digging into it, I am aware that testcase can influence the global
> state
> >> of
> >> a VM, for example, the resolution of class (both RI and Harmony have
> >> similar
> >> behavior). Although I changed the testcase as a workaround, it is not
> >> tested so thoroughly as I expected in order not to lead other tests to
> >> fail.
> >
> >
> > If a test's execution influence of VM state and this is critical for
> other
> > test then the test can fork VM (via Support_Exec.execJava()) and do all
> > testing in the forked VM.
>
> +1 -- and this should be the exception, in general tests should put
> things back as they found them. exec'ing a new Java is for those cases
> where you cannot do that.
>
> Regards,
> Tim
>
> --
>
> Tim Ellison (t.p.ellison@gmail.com)
> IBM Java technology centre, UK.
>
--
Leo Li
China Software Development Lab, IBM
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Stepan Mishura <st...@gmail.com>.
On 11/28/06, Tim Ellison wrote:
>
> Stepan Mishura wrote:
> > On 11/27/06, Leo Li wrote:
> >>
> >> Hi, all:
> >> During fixing the bug of Harmony-2249, I found that the testcase in
> >> one
> >> junit test file might lead to other fail in a different junit file.
> After
> >> digging into it, I am aware that testcase can influence the global
> state
> >> of
> >> a VM, for example, the resolution of class (both RI and Harmony have
> >> similar
> >> behavior). Although I changed the testcase as a workaround, it is not
> >> tested so thoroughly as I expected in order not to lead other tests to
> >> fail.
> >
> >
> > If a test's execution influence of VM state and this is critical for
> other
> > test then the test can fork VM (via Support_Exec.execJava()) and do all
> > testing in the forked VM.
>
> +1 -- and this should be the exception, in general tests should put
> things back as they found them. exec'ing a new Java is for those cases
> where you cannot do that.
Yes, exactly! We should use only is a test "taints" testing env. and we can
not restore it.
Thanks,
Stepan.
Regards,
> Tim
>
> --
>
> Tim Ellison (t.p.ellison@gmail.com)
> IBM Java technology centre, UK.
>
Re: [general][tests]To restrict the side-effect of junit tests by
launching it in a different VM.
Posted by Tim Ellison <t....@gmail.com>.
Stepan Mishura wrote:
> On 11/27/06, Leo Li wrote:
>>
>> Hi, all:
>> During fixing the bug of Harmony-2249, I found that the testcase in
>> one
>> junit test file might lead to other fail in a different junit file. After
>> digging into it, I am aware that testcase can influence the global state
>> of
>> a VM, for example, the resolution of class (both RI and Harmony have
>> similar
>> behavior). Although I changed the testcase as a workaround, it is not
>> tested so thoroughly as I expected in order not to lead other tests to
>> fail.
>
>
> If a test's execution influence of VM state and this is critical for other
> test then the test can fork VM (via Support_Exec.execJava()) and do all
> testing in the forked VM.
+1 -- and this should be the exception, in general tests should put
things back as they found them. exec'ing a new Java is for those cases
where you cannot do that.
Regards,
Tim
--
Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.
Re: [general][tests]To restrict the side-effect of junit tests by launching it in a different VM.
Posted by Stepan Mishura <st...@gmail.com>.
On 11/27/06, Leo Li wrote:
>
> Hi, all:
> During fixing the bug of Harmony-2249, I found that the testcase in
> one
> junit test file might lead to other fail in a different junit file. After
> digging into it, I am aware that testcase can influence the global state
> of
> a VM, for example, the resolution of class (both RI and Harmony have
> similar
> behavior). Although I changed the testcase as a workaround, it is not
> tested so thoroughly as I expected in order not to lead other tests to
> fail.
If a test's execution influence of VM state and this is critical for other
test then the test can fork VM (via Support_Exec.execJava()) and do all
testing in the forked VM.
Thanks,
Stepan.
So I recommend if we can specify some testcase that should run in junit
> with fork = "true". Maybe to add some annotation is one resolution or to
> add
> them to a property file as we done with excluded ones.
> Any suggestion?
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>
--
Stepan Mishura
Intel Middleware Products Division
------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org