You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Stepan Mishura <st...@gmail.com> on 2006/04/11 06:18:28 UTC
classlib test suite status emails?
Hi,
I've checked out at morning last updates, built the code base and run the
tests …and there are 24 tests failures!
There are 9 tests failures in
org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I saw these
failures before from time to time. It seems that tests depend on some race
conditions because they may pass if I rerun them. Paulex, are these tests
passing for you?
And there are new 15 test failures. So now if I'll make a code update or a
bug fix how I can be 100% sure that I don't do any regression?
Currently if a commit breaks the Harmony classlib build a notification with
subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be send to
harmony-commits mailing list. Is it possible to have the same for tests? I
mean that after completing automatic build the existing classlib tests suite
should be run. If there are failing tests then a report notification with
corresponding subject will be send.
Thanks,
Stepan Mishura
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
On 4/19/06, Stepan Mishura <st...@gmail.com> wrote:
> On 4/18/06, Mark Hindess wrote:
>
> > On 4/13/06, Stepan Mishura wrote:
> > >
> > > Hi Mark,
> > >
> > > HARMONY-331 was commited to the trunk so is there any chance to have
> > > classlib test suite status emails sent to the commits list?
> >
> > Ok. I've added notifiers to both our linux and windows jobs that are
> > doing build and test.
>
>
> Great!
>
> Unfortunately, since Continuum truncates notification messages, it
> > wont always be obvious what has broken. That said, the logs are a
> > little shorter since removed the "second stage" build with IBM VME,
> > Harmony and eclipse compiler step from the builds.
>
>
> May be it makes sense to broke intentionally the build (to force some tests
> to fail) and see what notification will be sent? Then everybody will see how
> it looks like.
You don't think we'll break things soon enough? ;-)
As it happens, the windows build tripped up on the unstable
SocketChannelTest test so it has failed. Sadly the mail didn't get
through, but I think I know why and I've fixed it. We should see a
success message when the test next passes.
-Mark.
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Stepan Mishura <st...@gmail.com>.
On 4/18/06, Mark Hindess wrote:
> On 4/13/06, Stepan Mishura wrote:
> >
> > Hi Mark,
> >
> > HARMONY-331 was commited to the trunk so is there any chance to have
> > classlib test suite status emails sent to the commits list?
>
> Ok. I've added notifiers to both our linux and windows jobs that are
> doing build and test.
Great!
Unfortunately, since Continuum truncates notification messages, it
> wont always be obvious what has broken. That said, the logs are a
> little shorter since removed the "second stage" build with IBM VME,
> Harmony and eclipse compiler step from the builds.
May be it makes sense to broke intentionally the build (to force some tests
to fail) and see what notification will be sent? Then everybody will see how
it looks like.
Thanks,
Stepan.
Regards,
> Mark.
>
> --
> Mark Hindess <ma...@googlemail.com>
> IBM Java Technology Centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
-----------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Thanks,
Stepan Mishura
Intel Middleware Products Division
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
On 4/13/06, Stepan Mishura <st...@gmail.com> wrote:
>
> Hi Mark,
>
> HARMONY-331 was commited to the trunk so is there any chance to have
> classlib test suite status emails sent to the commits list?
Ok. I've added notifiers to both our linux and windows jobs that are
doing build and test.
Unfortunately, since Continuum truncates notification messages, it
wont always be obvious what has broken. That said, the logs are a
little shorter since removed the "second stage" build with IBM VME,
Harmony and eclipse compiler step from the builds.
Regards,
Mark.
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Stepan Mishura <st...@gmail.com>.
On 4/11/06, Mark Hindess wrote:
>
> I've submitted a JIRA containing a fix that is something close to what
> Stepan suggested. Having module test targets append the name of the
> module to a build/test_report/test.errors (or test.failures) and the
> top level target fails if those files exist at the end of the run.
Hi Mark,
HARMONY-331 was commited to the trunk so is there any chance to have
classlib test suite status emails sent to the commits list?
Thanks,
Stepan.
The diff to make this change would be so much easier if we'd refactor
> the build files - perhaps as I suggested in HARMONY-293. I'm going to
> update this JIRA to fix it with respect to all the recent moves and
> test additions but I'd be interested in peoples views on this in the
> meantime. I think refactoring will make life much easier.
>
> -Mark.
>
> On 4/11/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> >
> >
> > Mark Hindess wrote:
> > > Yes. I was using the failureproperty mechanism. Trying to get this
> > > property propogated back to the top level ant file was what I was
> > > having trouble with.
> > >
> >
> > I had the same problem. I wanted the whole thing to halt on any
> > failure, and it didn't work... the top level ant script didn't pay
> > attention...
> >
> >
> > > Using a file as you suggest might help. I'll give that a try
> shortly...
> > >
> > > Incidentally, I'm seeing 12 failures and 3 errors on r393111. (And
> > > there are typos - "mathc" should be "match" - in the failure messages
> > > for java.security.cert testMatch and testClone.)
> > >
> > > Regards,
> > > Mark.
> > >
> > > On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > >> On 4/11/06, Mark Hindess wrote:
> > >>
> > >>> Stepan,
> > >>>
> > >>> I have another build running (but without notifications going to the
> > >>> list) that runs:
> > >>>
> > >>> 1) build (with reference jdk)
> > >>> 2) build with what we created with 1)
> > >>> 3) test
> > >>> 4) create classlists and compare with class load data for
> applications
> > >>> (tomcat, geronimo, continuum, etc.)
> > >>> 5) generate JAPI results
> > >>>
> > >>> I'd like to fail this build at step 3, but I can't see an easy way
> to
> > >>> get 'ant -f make/build.xml test' to run all tests and then fail if
> any
> > >>> of the module test sub-targets had test failures. I could parse the
> > >>> output I suppose, but I'd rather get ant to propagate the junit
> tasks
> > >>> failure property back up to the top level. I've tried a couple of
> > >>> things and none seem to work. Any suggestions welcome.
> > >>
> > >> Mark, did you try failureproperty parameter for junit task?
> > >> We may add it to ant sub-targets to raise a flag, for example, create
> file "
> > >> TESTS.FAILED" in the root, when tests for some module fail. And in
> the end
> > >> of tests suite run check whether this file exists on not.
> > >>
> > >> Thanks,
> > >> Stepan.
> > >>
> > >>
> > >> Regards,
> > >>> Mark.
> > >>>
> > >>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > >>>> Hi,
> > >>>>
> > >>>> I've checked out at morning last updates, built the code base and
> run
> > >>> the
> > >>>> tests …and there are 24 tests failures!
> > >>>>
> > >>>> There are 9 tests failures in
> > >>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I
> saw
> > >>> these
> > >>>> failures before from time to time. It seems that tests depend on
> some
> > >>> race
> > >>>> conditions because they may pass if I rerun them. Paulex, are these
> > >>> tests
> > >>>> passing for you?
> > >>>>
> > >>>> And there are new 15 test failures. So now if I'll make a code
> update
> > >>> or a
> > >>>> bug fix how I can be 100% sure that I don't do any regression?
> > >>>>
> > >>>> Currently if a commit breaks the Harmony classlib build a
> notification
> > >>> with
> > >>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be
> send
> > >>> to
> > >>>> harmony-commits mailing list. Is it possible to have the same for
> tests?
> > >>> I
> > >>>> mean that after completing automatic build the existing classlib
> tests
> > >>> suite
> > >>>> should be run. If there are failing tests then a report
> notification
> > >>> with
> > >>>> corresponding subject will be send.
> > >>>>
> > >>>> Thanks,
> > >>>> Stepan Mishura
> > >>>> Intel Middleware Products Division
> > >>> --
> > >>> Mark Hindess <ma...@googlemail.com>
> > >>> IBM Java Technology Centre, UK.
> > >>>
> > >>>
> ---------------------------------------------------------------------
> > >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >>> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > >>>
> > >>>
> > >>
> > >> --
> > >> ---------------------------------------------------------------------
> > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > >>
> > >> Thanks,
> > >> Stepan Mishura
> > >> Intel Middleware Products Division
> > >>
> > >>
> > >
> > >
> > > --
> > > Mark Hindess <ma...@googlemail.com>
> > > IBM Java Technology Centre, UK.
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> --
> Mark Hindess <ma...@googlemail.com>
> IBM Java Technology Centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Thanks,
Stepan Mishura
Intel Middleware Products Division
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
I've submitted a JIRA containing a fix that is something close to what
Stepan suggested. Having module test targets append the name of the
module to a build/test_report/test.errors (or test.failures) and the
top level target fails if those files exist at the end of the run.
The diff to make this change would be so much easier if we'd refactor
the build files - perhaps as I suggested in HARMONY-293. I'm going to
update this JIRA to fix it with respect to all the recent moves and
test additions but I'd be interested in peoples views on this in the
meantime. I think refactoring will make life much easier.
-Mark.
On 4/11/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
>
>
> Mark Hindess wrote:
> > Yes. I was using the failureproperty mechanism. Trying to get this
> > property propogated back to the top level ant file was what I was
> > having trouble with.
> >
>
> I had the same problem. I wanted the whole thing to halt on any
> failure, and it didn't work... the top level ant script didn't pay
> attention...
>
>
> > Using a file as you suggest might help. I'll give that a try shortly...
> >
> > Incidentally, I'm seeing 12 failures and 3 errors on r393111. (And
> > there are typos - "mathc" should be "match" - in the failure messages
> > for java.security.cert testMatch and testClone.)
> >
> > Regards,
> > Mark.
> >
> > On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> >> On 4/11/06, Mark Hindess wrote:
> >>
> >>> Stepan,
> >>>
> >>> I have another build running (but without notifications going to the
> >>> list) that runs:
> >>>
> >>> 1) build (with reference jdk)
> >>> 2) build with what we created with 1)
> >>> 3) test
> >>> 4) create classlists and compare with class load data for applications
> >>> (tomcat, geronimo, continuum, etc.)
> >>> 5) generate JAPI results
> >>>
> >>> I'd like to fail this build at step 3, but I can't see an easy way to
> >>> get 'ant -f make/build.xml test' to run all tests and then fail if any
> >>> of the module test sub-targets had test failures. I could parse the
> >>> output I suppose, but I'd rather get ant to propagate the junit tasks
> >>> failure property back up to the top level. I've tried a couple of
> >>> things and none seem to work. Any suggestions welcome.
> >>
> >> Mark, did you try failureproperty parameter for junit task?
> >> We may add it to ant sub-targets to raise a flag, for example, create file "
> >> TESTS.FAILED" in the root, when tests for some module fail. And in the end
> >> of tests suite run check whether this file exists on not.
> >>
> >> Thanks,
> >> Stepan.
> >>
> >>
> >> Regards,
> >>> Mark.
> >>>
> >>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> >>>> Hi,
> >>>>
> >>>> I've checked out at morning last updates, built the code base and run
> >>> the
> >>>> tests …and there are 24 tests failures!
> >>>>
> >>>> There are 9 tests failures in
> >>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I saw
> >>> these
> >>>> failures before from time to time. It seems that tests depend on some
> >>> race
> >>>> conditions because they may pass if I rerun them. Paulex, are these
> >>> tests
> >>>> passing for you?
> >>>>
> >>>> And there are new 15 test failures. So now if I'll make a code update
> >>> or a
> >>>> bug fix how I can be 100% sure that I don't do any regression?
> >>>>
> >>>> Currently if a commit breaks the Harmony classlib build a notification
> >>> with
> >>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be send
> >>> to
> >>>> harmony-commits mailing list. Is it possible to have the same for tests?
> >>> I
> >>>> mean that after completing automatic build the existing classlib tests
> >>> suite
> >>>> should be run. If there are failing tests then a report notification
> >>> with
> >>>> corresponding subject will be send.
> >>>>
> >>>> Thanks,
> >>>> Stepan Mishura
> >>>> Intel Middleware Products Division
> >>> --
> >>> Mark Hindess <ma...@googlemail.com>
> >>> IBM Java Technology Centre, UK.
> >>>
> >>> ---------------------------------------------------------------------
> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>
> >>>
> >>
> >> --
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >> Thanks,
> >> Stepan Mishura
> >> Intel Middleware Products Division
> >>
> >>
> >
> >
> > --
> > Mark Hindess <ma...@googlemail.com>
> > IBM Java Technology Centre, UK.
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
Mark Hindess wrote:
> Yes. I was using the failureproperty mechanism. Trying to get this
> property propogated back to the top level ant file was what I was
> having trouble with.
>
I had the same problem. I wanted the whole thing to halt on any
failure, and it didn't work... the top level ant script didn't pay
attention...
> Using a file as you suggest might help. I'll give that a try shortly...
>
> Incidentally, I'm seeing 12 failures and 3 errors on r393111. (And
> there are typos - "mathc" should be "match" - in the failure messages
> for java.security.cert testMatch and testClone.)
>
> Regards,
> Mark.
>
> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>> On 4/11/06, Mark Hindess wrote:
>>
>>> Stepan,
>>>
>>> I have another build running (but without notifications going to the
>>> list) that runs:
>>>
>>> 1) build (with reference jdk)
>>> 2) build with what we created with 1)
>>> 3) test
>>> 4) create classlists and compare with class load data for applications
>>> (tomcat, geronimo, continuum, etc.)
>>> 5) generate JAPI results
>>>
>>> I'd like to fail this build at step 3, but I can't see an easy way to
>>> get 'ant -f make/build.xml test' to run all tests and then fail if any
>>> of the module test sub-targets had test failures. I could parse the
>>> output I suppose, but I'd rather get ant to propagate the junit tasks
>>> failure property back up to the top level. I've tried a couple of
>>> things and none seem to work. Any suggestions welcome.
>>
>> Mark, did you try failureproperty parameter for junit task?
>> We may add it to ant sub-targets to raise a flag, for example, create file "
>> TESTS.FAILED" in the root, when tests for some module fail. And in the end
>> of tests suite run check whether this file exists on not.
>>
>> Thanks,
>> Stepan.
>>
>>
>> Regards,
>>> Mark.
>>>
>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I've checked out at morning last updates, built the code base and run
>>> the
>>>> tests …and there are 24 tests failures!
>>>>
>>>> There are 9 tests failures in
>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I saw
>>> these
>>>> failures before from time to time. It seems that tests depend on some
>>> race
>>>> conditions because they may pass if I rerun them. Paulex, are these
>>> tests
>>>> passing for you?
>>>>
>>>> And there are new 15 test failures. So now if I'll make a code update
>>> or a
>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>
>>>> Currently if a commit breaks the Harmony classlib build a notification
>>> with
>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be send
>>> to
>>>> harmony-commits mailing list. Is it possible to have the same for tests?
>>> I
>>>> mean that after completing automatic build the existing classlib tests
>>> suite
>>>> should be run. If there are failing tests then a report notification
>>> with
>>>> corresponding subject will be send.
>>>>
>>>> Thanks,
>>>> Stepan Mishura
>>>> Intel Middleware Products Division
>>> --
>>> Mark Hindess <ma...@googlemail.com>
>>> IBM Java Technology Centre, UK.
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>
>> --
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>> Thanks,
>> Stepan Mishura
>> Intel Middleware Products Division
>>
>>
>
>
> --
> Mark Hindess <ma...@googlemail.com>
> IBM Java Technology Centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by George Harley <ge...@googlemail.com>.
Mark Hindess wrote:
> George is taking a look at the ones I mentioned - which are from our
> Linux build.
>
> I see quite a few more (including DatagramChannelTest) on our windows
> build.
Our internal Windows build has not been clean for a while now. I am
currently not looking at anything beyond the new failures that emerged
after my commit activity last night.
Best regards,
George
> Paulex can see the test report for this machine so hopefully
> that will help him track down the cause of these failures.
>
> Regards,
> Mark.
>
> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>
>> The same for me + DatagramChannelTest
>>
>> Thanks,
>> Stepan.
>>
>> On 4/11/06, Mark Hindess wrote:
>>
>>> No. These:
>>>
>>> F
>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>> E
>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>> E
>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>> E
>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>> F java.security.cert.X509CertSelectorTest.testMatch
>>> F java.security.cert.X509CertSelectorTest.testClone
>>> F tests.api.java.sql.DateTest.testSetTimelong
>>> F tests.api.java.sql.DateTest.testToString
>>> F tests.api.java.sql.DateTest.testValueOf
>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>> F tests.api.java.sql.TimestampTest.testToString
>>> F tests.api.java.util.DateTest.test_toGMTString
>>> F tests.api.java.util.DateTest.test_toString
>>>
>>> Regards,
>>> Mark.
>>>
>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>
>>>> On 4/11/06, Mark Hindess wrote:
>>>>
>>>>> Yes. I was using the failureproperty mechanism. Trying to get this
>>>>> property propogated back to the top level ant file was what I was
>>>>> having trouble with.
>>>>>
>>>>> Using a file as you suggest might help. I'll give that a try
>>>>>
>>> shortly...
>>>
>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>
>>>> I guess that you have 9 tests from DatagramChannelTest passed. And 12 +
>>>>
>>> 3 =
>>>
>>>> 15 :-)
>>>>
>>>> (And
>>>>
>>>>> there are typos - "mathc" should be "match" - in the failure messages
>>>>> for java.security.cert testMatch and testClone.)
>>>>>
>>>> I've fixed typo.
>>>>
>>>> Thanks,
>>>> Stepan.
>>>>
>>>> Regards,
>>>>
>>>>> Mark.
>>>>>
>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>
>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>
>>>>>>
>>>>>>> Stepan,
>>>>>>>
>>>>>>> I have another build running (but without notifications going to
>>>>>>>
>>> the
>>>
>>>>>>> list) that runs:
>>>>>>>
>>>>>>> 1) build (with reference jdk)
>>>>>>> 2) build with what we created with 1)
>>>>>>> 3) test
>>>>>>> 4) create classlists and compare with class load data for
>>>>>>>
>>> applications
>>>
>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>> 5) generate JAPI results
>>>>>>>
>>>>>>> I'd like to fail this build at step 3, but I can't see an easy way
>>>>>>>
>>> to
>>>
>>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail if
>>>>>>>
>>> any
>>>
>>>>>>> of the module test sub-targets had test failures. I could parse
>>>>>>>
>>> the
>>>
>>>>>>> output I suppose, but I'd rather get ant to propagate the junit
>>>>>>>
>>> tasks
>>>
>>>>>>> failure property back up to the top level. I've tried a couple of
>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>
>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>>>
>>> create
>>>
>>>>> file "
>>>>>
>>>>>> TESTS.FAILED" in the root, when tests for some module fail. And in
>>>>>>
>>> the
>>>
>>>>> end
>>>>>
>>>>>> of tests suite run check whether this file exists on not.
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan.
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>>> Mark.
>>>>>>>
>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I've checked out at morning last updates, built the code base
>>>>>>>>
>>> and
>>>
>>>>> run
>>>>>
>>>>>>> the
>>>>>>>
>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>
>>>>>>>> There are 9 tests failures in
>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest –
>>>>>>>>
>>> I
>>>
>>>>> saw
>>>>>
>>>>>>> these
>>>>>>>
>>>>>>>> failures before from time to time. It seems that tests depend on
>>>>>>>>
>>>>> some
>>>>>
>>>>>>> race
>>>>>>>
>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>>>>>
>>> these
>>>
>>>>>>> tests
>>>>>>>
>>>>>>>> passing for you?
>>>>>>>>
>>>>>>>> And there are new 15 test failures. So now if I'll make a code
>>>>>>>>
>>>>> update
>>>>>
>>>>>>> or a
>>>>>>>
>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>>
>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>>>
>>>>> notification
>>>>>
>>>>>>> with
>>>>>>>
>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
>>>>>>>>
>>> be
>>>
>>>>> send
>>>>>
>>>>>>> to
>>>>>>>
>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>>>>>
>>> for
>>>
>>>>> tests?
>>>>>
>>>>>>> I
>>>>>>>
>>>>>>>> mean that after completing automatic build the existing classlib
>>>>>>>>
>>>>> tests
>>>>>
>>>>>>> suite
>>>>>>>
>>>>>>>> should be run. If there are failing tests then a report
>>>>>>>>
>>> notification
>>>
>>>>>>> with
>>>>>>>
>>>>>>>> corresponding subject will be send.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan Mishura
>>>>>>>> Intel Middleware Products Division
>>>>>>>>
>>>>>>> --
>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>
>>>>>>>
>>>>>>>
>>> ---------------------------------------------------------------------
>>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail:
>>>>>>>
>>> harmony-dev-unsubscribe@incubator.apache.org
>>>
>>>>>>> For additional commands, e-mail:
>>>>>>>
>>> harmony-dev-help@incubator.apache.org
>>>
>>>>>>>
>>>>>> --
>>>>>>
>>>>>>
>>> ---------------------------------------------------------------------
>>>
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>>>>>
>>> harmony-dev-help@incubator.apache.org
>>>
>>>>>> Thanks,
>>>>>> Stepan Mishura
>>>>>> Intel Middleware Products Division
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> Mark Hindess <ma...@googlemail.com>
>>>>> IBM Java Technology Centre, UK.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>
>>>>>
>>>>>
>>>> --
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>> Thanks,
>>>> Stepan Mishura
>>>> Intel Middleware Products Division
>>>>
>>>>
>>>>
>>> --
>>> Mark Hindess <ma...@googlemail.com>
>>> IBM Java Technology Centre, UK.
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>>
>> --
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>> Thanks,
>> Stepan Mishura
>> Intel Middleware Products Division
>>
>>
>>
>
>
> --
> Mark Hindess <ma...@googlemail.com>
> IBM Java Technology Centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
George is taking a look at the ones I mentioned - which are from our
Linux build.
I see quite a few more (including DatagramChannelTest) on our windows
build. Paulex can see the test report for this machine so hopefully
that will help him track down the cause of these failures.
Regards,
Mark.
On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> The same for me + DatagramChannelTest
>
> Thanks,
> Stepan.
>
> On 4/11/06, Mark Hindess wrote:
> >
> > No. These:
> >
> > F
> > org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
> > E
> > org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
> > E
> > org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
> > F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
> > E
> > org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
> > F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
> > F java.security.cert.X509CertSelectorTest.testMatch
> > F java.security.cert.X509CertSelectorTest.testClone
> > F tests.api.java.sql.DateTest.testSetTimelong
> > F tests.api.java.sql.DateTest.testToString
> > F tests.api.java.sql.DateTest.testValueOf
> > F tests.api.java.sql.TimestampTest.testSetNanosint
> > F tests.api.java.sql.TimestampTest.testToString
> > F tests.api.java.util.DateTest.test_toGMTString
> > F tests.api.java.util.DateTest.test_toString
> >
> > Regards,
> > Mark.
> >
> > On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > On 4/11/06, Mark Hindess wrote:
> > > >
> > > > Yes. I was using the failureproperty mechanism. Trying to get this
> > > > property propogated back to the top level ant file was what I was
> > > > having trouble with.
> > > >
> > > > Using a file as you suggest might help. I'll give that a try
> > shortly...
> > > >
> > > > Incidentally, I'm seeing 12 failures and 3 errors on r393111.
> > >
> > >
> > > I guess that you have 9 tests from DatagramChannelTest passed. And 12 +
> > 3 =
> > > 15 :-)
> > >
> > > (And
> > > > there are typos - "mathc" should be "match" - in the failure messages
> > > > for java.security.cert testMatch and testClone.)
> > >
> > >
> > > I've fixed typo.
> > >
> > > Thanks,
> > > Stepan.
> > >
> > > Regards,
> > > > Mark.
> > > >
> > > > On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > > > On 4/11/06, Mark Hindess wrote:
> > > > >
> > > > > > Stepan,
> > > > > >
> > > > > > I have another build running (but without notifications going to
> > the
> > > > > > list) that runs:
> > > > > >
> > > > > > 1) build (with reference jdk)
> > > > > > 2) build with what we created with 1)
> > > > > > 3) test
> > > > > > 4) create classlists and compare with class load data for
> > applications
> > > > > > (tomcat, geronimo, continuum, etc.)
> > > > > > 5) generate JAPI results
> > > > > >
> > > > > > I'd like to fail this build at step 3, but I can't see an easy way
> > to
> > > > > > get 'ant -f make/build.xml test' to run all tests and then fail if
> > any
> > > > > > of the module test sub-targets had test failures. I could parse
> > the
> > > > > > output I suppose, but I'd rather get ant to propagate the junit
> > tasks
> > > > > > failure property back up to the top level. I've tried a couple of
> > > > > > things and none seem to work. Any suggestions welcome.
> > > > >
> > > > >
> > > > > Mark, did you try failureproperty parameter for junit task?
> > > > > We may add it to ant sub-targets to raise a flag, for example,
> > create
> > > > file "
> > > > > TESTS.FAILED" in the root, when tests for some module fail. And in
> > the
> > > > end
> > > > > of tests suite run check whether this file exists on not.
> > > > >
> > > > > Thanks,
> > > > > Stepan.
> > > > >
> > > > >
> > > > > Regards,
> > > > > > Mark.
> > > > > >
> > > > > > On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > I've checked out at morning last updates, built the code base
> > and
> > > > run
> > > > > > the
> > > > > > > tests …and there are 24 tests failures!
> > > > > > >
> > > > > > > There are 9 tests failures in
> > > > > > > org.apache.harmony.tests.java.nio.channels.DatagramChannelTest –
> > I
> > > > saw
> > > > > > these
> > > > > > > failures before from time to time. It seems that tests depend on
> > > > some
> > > > > > race
> > > > > > > conditions because they may pass if I rerun them. Paulex, are
> > these
> > > > > > tests
> > > > > > > passing for you?
> > > > > > >
> > > > > > > And there are new 15 test failures. So now if I'll make a code
> > > > update
> > > > > > or a
> > > > > > > bug fix how I can be 100% sure that I don't do any regression?
> > > > > > >
> > > > > > > Currently if a commit breaks the Harmony classlib build a
> > > > notification
> > > > > > with
> > > > > > > subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
> > be
> > > > send
> > > > > > to
> > > > > > > harmony-commits mailing list. Is it possible to have the same
> > for
> > > > tests?
> > > > > > I
> > > > > > > mean that after completing automatic build the existing classlib
> > > > tests
> > > > > > suite
> > > > > > > should be run. If there are failing tests then a report
> > notification
> > > > > > with
> > > > > > > corresponding subject will be send.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Stepan Mishura
> > > > > > > Intel Middleware Products Division
> > > > > >
> > > > > > --
> > > > > > Mark Hindess <ma...@googlemail.com>
> > > > > > IBM Java Technology Centre, UK.
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > To unsubscribe, e-mail:
> > harmony-dev-unsubscribe@incubator.apache.org
> > > > > > For additional commands, e-mail:
> > harmony-dev-help@incubator.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > ---------------------------------------------------------------------
> > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > For additional commands, e-mail:
> > harmony-dev-help@incubator.apache.org
> > > > >
> > > > > Thanks,
> > > > > Stepan Mishura
> > > > > Intel Middleware Products Division
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Mark Hindess <ma...@googlemail.com>
> > > > IBM Java Technology Centre, UK.
> > > >
> > > > ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > >
> > > --
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > > Thanks,
> > > Stepan Mishura
> > > Intel Middleware Products Division
> > >
> > >
> >
> >
> > --
> > Mark Hindess <ma...@googlemail.com>
> > IBM Java Technology Centre, UK.
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> --
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
>
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
Stepan Mishura wrote:
> On 4/11/06, Mark Hindess wrote:
>
>> Personally, obviously, I'd expect people to run the tests before
>> committing.
>>
>> However, I notice that since enabling the security tests - which fork
>> for every test - that the tests take over half an hour to run now on
>> our Linux build machine. So I can see why enthusiasm might lead to
>> people not running all the tests but instead perhaps just running
>> those from the module you are changing.
>
>
> I think that this message[1] relates to the problem with running big test
> suites. So should we do the similar thing i.e. define which test suites
> should be run for each module?
The problem is side effects (as George discovered...)
geir
>
> Thanks,
> Stepan.
>
> [1]
> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200603.mbox/%3c442C1C79.1060805@sourcelabs.com%3e
>
>
>
>> George just commented in a commit message backing out some changes in
>> text that all the text tests passed even though the changed caused
>> failures elsewhere.
>>
>> Regards,
>> Mark.
>>
>> On 4/11/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
>>> Just curious (and this isn't a criticism - I'm just as guilty of not
>>> doing this)...
>>>
>>> Don't you run the tests before committing?
>>>
>>> geir
>>>
>>> George Harley wrote:
>>>> Hi,
>>>>
>>>> It *seems* like things started failing after I committed the changes
>> for
>>>> HARMONY-205 last night. I'm looking into this now. If the
>> investigation
>>>> begins to take up too much time I will back the changes out.
>>>>
>>>> Best regards,
>>>> George
>>>>
>>>>
>>>> Stepan Mishura wrote:
>>>>> The same for me + DatagramChannelTest
>>>>>
>>>>> Thanks,
>>>>> Stepan.
>>>>>
>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>
>>>>>> No. These:
>>>>>>
>>>>>> F
>>>>>>
>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>>>>> E
>>>>>>
>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>>>>> E
>>>>>>
>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>>>> E
>>>>>>
>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>>>> F tests.api.java.sql.DateTest.testToString
>>>>>> F tests.api.java.sql.DateTest.testValueOf
>>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>>>> F tests.api.java.sql.TimestampTest.testToString
>>>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>>>> F tests.api.java.util.DateTest.test_toString
>>>>>>
>>>>>> Regards,
>>>>>> Mark.
>>>>>>
>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>
>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>
>>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get
>> this
>>>>>>>> property propogated back to the top level ant file was what I was
>>>>>>>> having trouble with.
>>>>>>>>
>>>>>>>> Using a file as you suggest might help. I'll give that a try
>>>>>>>>
>>>>>> shortly...
>>>>>>
>>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>>>>
>>>>>>> I guess that you have 9 tests from DatagramChannelTest passed. And
>>>>>>> 12 +
>>>>>>>
>>>>>> 3 =
>>>>>>
>>>>>>> 15 :-)
>>>>>>>
>>>>>>> (And
>>>>>>>
>>>>>>>> there are typos - "mathc" should be "match" - in the failure
>> messages
>>>>>>>> for java.security.cert testMatch and testClone.)
>>>>>>>>
>>>>>>> I've fixed typo.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Stepan.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>>> Mark.
>>>>>>>>
>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Stepan,
>>>>>>>>>>
>>>>>>>>>> I have another build running (but without notifications going to
>>>>>>>>>>
>>>>>> the
>>>>>>
>>>>>>>>>> list) that runs:
>>>>>>>>>>
>>>>>>>>>> 1) build (with reference jdk)
>>>>>>>>>> 2) build with what we created with 1)
>>>>>>>>>> 3) test
>>>>>>>>>> 4) create classlists and compare with class load data for
>>>>>>>>>>
>>>>>> applications
>>>>>>
>>>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>>>> 5) generate JAPI results
>>>>>>>>>>
>>>>>>>>>> I'd like to fail this build at step 3, but I can't see an easy
>> way
>>>>>> to
>>>>>>
>>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail
>> if
>>>>>> any
>>>>>>
>>>>>>>>>> of the module test sub-targets had test failures. I could parse
>>>>>>>>>>
>>>>>> the
>>>>>>
>>>>>>>>>> output I suppose, but I'd rather get ant to propagate the junit
>>>>>>>>>>
>>>>>> tasks
>>>>>>
>>>>>>>>>> failure property back up to the top level. I've tried a couple
>> of
>>>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>>>>
>>>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>>>>>>
>>>>>> create
>>>>>>
>>>>>>>> file "
>>>>>>>>
>>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail. And
>> in
>>>>>> the
>>>>>>
>>>>>>>> end
>>>>>>>>
>>>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Stepan.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>>> Mark.
>>>>>>>>>>
>>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I've checked out at morning last updates, built the code base
>>>>>>>>>>>
>>>>>> and
>>>>>>
>>>>>>>> run
>>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>>>
>>>>>>>>>>> There are 9 tests failures in
>>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest–
>>>>>>>>>>>
>>>>>> I
>>>>>>
>>>>>>>> saw
>>>>>>>>
>>>>>>>>>> these
>>>>>>>>>>
>>>>>>>>>>> failures before from time to time. It seems that tests depend
>> on
>>>>>>>> some
>>>>>>>>
>>>>>>>>>> race
>>>>>>>>>>
>>>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>>>>>>>>
>>>>>> these
>>>>>>
>>>>>>>>>> tests
>>>>>>>>>>
>>>>>>>>>>> passing for you?
>>>>>>>>>>>
>>>>>>>>>>> And there are new 15 test failures. So now if I'll make a code
>>>>>>>>>>>
>>>>>>>> update
>>>>>>>>
>>>>>>>>>> or a
>>>>>>>>>>
>>>>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>>>>>
>>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>>>>>>
>>>>>>>> notification
>>>>>>>>
>>>>>>>>>> with
>>>>>>>>>>
>>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
>>>>>>>>>>>
>>>>>> be
>>>>>>
>>>>>>>> send
>>>>>>>>
>>>>>>>>>> to
>>>>>>>>>>
>>>>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>>>>>>>>
>>>>>> for
>>>>>>
>>>>>>>> tests?
>>>>>>>>
>>>>>>>>>> I
>>>>>>>>>>
>>>>>>>>>>> mean that after completing automatic build the existing
>> classlib
>>>>>>>> tests
>>>>>>>>
>>>>>>>>>> suite
>>>>>>>>>>
>>>>>>>>>>> should be run. If there are failing tests then a report
>>>>>>>>>>>
>>>>>> notification
>>>>>>
>>>>>>>>>> with
>>>>>>>>>>
>>>>>>>>>>> corresponding subject will be send.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Stepan Mishura
>>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>
>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>
>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail:
>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Stepan Mishura
>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>
>>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail:
>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>> harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail:
>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>> harmony-dev-help@incubator.apache.org
>>>>>>> Thanks,
>>>>>>> Stepan Mishura
>>>>>>> Intel Middleware Products Division
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>> IBM Java Technology Centre, UK.
>>>>>>
>>>>>>
>> ---------------------------------------------------------------------
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail:
>> harmony-dev-help@incubator.apache.org
>>>>> Thanks,
>>>>> Stepan Mishura
>>>>> Intel Middleware Products Division
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>
>> --
>> Mark Hindess <ma...@googlemail.com>
>> IBM Java Technology Centre, UK.
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
>
> --
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Mikhail Loenko <ml...@gmail.com>.
Well I did not suggest maintaining separate set of manifests :)
Can we somehow test validity of [existing] manifests?
Thanks,
Mikhail
2006/4/12, Mark Hindess <ma...@googlemail.com>:
> Mikhail,
>
> While that is true, maintaining separate dependency information is
> just as (un)likely to be correct. I think we should use the manifests
> and fix them if they are inaccurate.
>
> Regards,
> Mark.
>
>
> On 4/12/06, Mikhail Loenko <ml...@gmail.com> wrote:
> > BTW AFAIK we do not test whether manifests list all imports/exports
> >
> > So if we create data from manifests then it could be insufficient and
> > we might have the same failures we had yesterday.
> >
> > Thanks,
> > Mikhail
> >
> > 2006/4/12, Anton Avtamonov <an...@gmail.com>:
> > > On 4/12/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> > > > Why have a static file? Why not just generate from the modules
> > > > manifests? Then the error can only be in one place - the dependent
> > > > module... right?
> > >
> > > If they have enough info - of course! That would be excellent.
> > > I suppose it is a bit not trivial since the required information is
> > > not the modules the particular module is depended on, but all the
> > > modules which depend on the particular module. However it should be
> > > not hard to revert.
> > > Did I catch your idea right?
> > >
> > > --
> > > Anton Avtamonov,
> > > Intel Middleware Products Division
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> --
> Mark Hindess <ma...@googlemail.com>
> IBM Java Technology Centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Anton Avtamonov <an...@gmail.com>.
Well, you will need to build-up dependency info anyway, right?
Therefore probably that would be a good chance to revise and update
manifests :-)?
--
Anton Avtamonov,
Intel Middleware Products Division
On 4/12/06, Mikhail Loenko <ml...@gmail.com> wrote:
> BTW AFAIK we do not test whether manifests list all imports/exports
>
> So if we create data from manifests then it could be insufficient and
> we might have the same failures we had yesterday.
>
> Thanks,
> Mikhail
>
> 2006/4/12, Anton Avtamonov <an...@gmail.com>:
> > On 4/12/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> > > Why have a static file? Why not just generate from the modules
> > > manifests? Then the error can only be in one place - the dependent
> > > module... right?
> >
> > If they have enough info - of course! That would be excellent.
> > I suppose it is a bit not trivial since the required information is
> > not the modules the particular module is depended on, but all the
> > modules which depend on the particular module. However it should be
> > not hard to revert.
> > Did I catch your idea right?
> >
> > --
> > Anton Avtamonov,
> > Intel Middleware Products Division
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
Mikhail,
While that is true, maintaining separate dependency information is
just as (un)likely to be correct. I think we should use the manifests
and fix them if they are inaccurate.
Regards,
Mark.
On 4/12/06, Mikhail Loenko <ml...@gmail.com> wrote:
> BTW AFAIK we do not test whether manifests list all imports/exports
>
> So if we create data from manifests then it could be insufficient and
> we might have the same failures we had yesterday.
>
> Thanks,
> Mikhail
>
> 2006/4/12, Anton Avtamonov <an...@gmail.com>:
> > On 4/12/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> > > Why have a static file? Why not just generate from the modules
> > > manifests? Then the error can only be in one place - the dependent
> > > module... right?
> >
> > If they have enough info - of course! That would be excellent.
> > I suppose it is a bit not trivial since the required information is
> > not the modules the particular module is depended on, but all the
> > modules which depend on the particular module. However it should be
> > not hard to revert.
> > Did I catch your idea right?
> >
> > --
> > Anton Avtamonov,
> > Intel Middleware Products Division
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
Mikhail Loenko wrote:
> BTW AFAIK we do not test whether manifests list all imports/exports
>
> So if we create data from manifests then it could be insufficient and
> we might have the same failures we had yesterday.
Yes... but we already have the manifests. They have to be right for OSGi.
So given that we need to have this dependency information, and have it
be correct, for the manifests, why not focus on getting that right, and
forever more leverage that beauty, goodness and truth?
geir
>
> Thanks,
> Mikhail
>
> 2006/4/12, Anton Avtamonov <an...@gmail.com>:
>> On 4/12/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
>>> Why have a static file? Why not just generate from the modules
>>> manifests? Then the error can only be in one place - the dependent
>>> module... right?
>> If they have enough info - of course! That would be excellent.
>> I suppose it is a bit not trivial since the required information is
>> not the modules the particular module is depended on, but all the
>> modules which depend on the particular module. However it should be
>> not hard to revert.
>> Did I catch your idea right?
>>
>> --
>> Anton Avtamonov,
>> Intel Middleware Products Division
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Mikhail Loenko <ml...@gmail.com>.
BTW AFAIK we do not test whether manifests list all imports/exports
So if we create data from manifests then it could be insufficient and
we might have the same failures we had yesterday.
Thanks,
Mikhail
2006/4/12, Anton Avtamonov <an...@gmail.com>:
> On 4/12/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> > Why have a static file? Why not just generate from the modules
> > manifests? Then the error can only be in one place - the dependent
> > module... right?
>
> If they have enough info - of course! That would be excellent.
> I suppose it is a bit not trivial since the required information is
> not the modules the particular module is depended on, but all the
> modules which depend on the particular module. However it should be
> not hard to revert.
> Did I catch your idea right?
>
> --
> Anton Avtamonov,
> Intel Middleware Products Division
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Anton Avtamonov <an...@gmail.com>.
On 4/12/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> Why have a static file? Why not just generate from the modules
> manifests? Then the error can only be in one place - the dependent
> module... right?
If they have enough info - of course! That would be excellent.
I suppose it is a bit not trivial since the required information is
not the modules the particular module is depended on, but all the
modules which depend on the particular module. However it should be
not hard to revert.
Did I catch your idea right?
--
Anton Avtamonov,
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
I think the dependencies for a module belong under the module
directory not at the top level but I agree it might be nice to find a
way to pull them out of the build.xml files.
Regards,
Mark.
On 4/12/06, Anton Avtamonov <an...@gmail.com> wrote:
> On 4/12/06, George Harley <ge...@googlemail.com> wrote:
> > Hi,
> >
> > Thanks for your comments.
> >
> > Working on the "walk first then try to run" principle, I will try to
> > start adding in "run.dependants.tests" targets to individual modules'
> > build.xml scripts over the next few days. Text seems like a good
> > candidate to start.
>
> George,
> Just a raw idea: it is possible to have description of all
> dependencies located in a single place out of individual modules? As
> XML file for instance?
>
> As I understoond you are going to list all the dependencies right in
> the build.xml of each module. IMHO, such approach makes not too easy
> to understand the dependencies. I would suppose to have some XML file
> representing kind of dependency table which is accessed by the
> build.xml of each module (by some specific target.
> run.dependants.tests is fine) to retreive the particular list of
> dependencies for the particular module.
> Having all the dependencies in the single place would simplify their
> support. Besides, I expect it would be not too difficult to create
> XSLT to generate HTML table in the perfect human-readable form...
>
> As I said that is just an idea. I'm not experienced in ant-scripting,
> not sure if that is good idea and how it difficult to implement.
>
>
> Wishes,
> --
> Anton Avtamonov,
> Intel Middleware Products Division
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
Why have a static file? Why not just generate from the modules
manifests? Then the error can only be in one place - the dependent
module... right?
geir
George Harley wrote:
> Hi Anton,
>
> It is a good idea if it helps developers build up a map of how the class
> library modules relate to one another. I am not currently sure how such
> a top level description file can be incorporated into the kind of scheme
> I was thinking about. I will certainly look for opportunities to factor
> this information out to a higher level if possible.
>
> Best regards,
> George
>
>
> Anton Avtamonov wrote:
>> On 4/12/06, George Harley <ge...@googlemail.com> wrote:
>>
>>> Hi,
>>>
>>> Thanks for your comments.
>>>
>>> Working on the "walk first then try to run" principle, I will try to
>>> start adding in "run.dependants.tests" targets to individual modules'
>>> build.xml scripts over the next few days. Text seems like a good
>>> candidate to start.
>>>
>>
>> George,
>> Just a raw idea: it is possible to have description of all
>> dependencies located in a single place out of individual modules? As
>> XML file for instance?
>>
>> As I understoond you are going to list all the dependencies right in
>> the build.xml of each module. IMHO, such approach makes not too easy
>> to understand the dependencies. I would suppose to have some XML file
>> representing kind of dependency table which is accessed by the
>> build.xml of each module (by some specific target.
>> run.dependants.tests is fine) to retreive the particular list of
>> dependencies for the particular module.
>> Having all the dependencies in the single place would simplify their
>> support. Besides, I expect it would be not too difficult to create
>> XSLT to generate HTML table in the perfect human-readable form...
>>
>> As I said that is just an idea. I'm not experienced in ant-scripting,
>> not sure if that is good idea and how it difficult to implement.
>>
>>
>> Wishes,
>> --
>> Anton Avtamonov,
>> Intel Middleware Products Division
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by George Harley <ge...@googlemail.com>.
Hi Anton,
It is a good idea if it helps developers build up a map of how the class
library modules relate to one another. I am not currently sure how such
a top level description file can be incorporated into the kind of scheme
I was thinking about. I will certainly look for opportunities to factor
this information out to a higher level if possible.
Best regards,
George
Anton Avtamonov wrote:
> On 4/12/06, George Harley <ge...@googlemail.com> wrote:
>
>> Hi,
>>
>> Thanks for your comments.
>>
>> Working on the "walk first then try to run" principle, I will try to
>> start adding in "run.dependants.tests" targets to individual modules'
>> build.xml scripts over the next few days. Text seems like a good
>> candidate to start.
>>
>
> George,
> Just a raw idea: it is possible to have description of all
> dependencies located in a single place out of individual modules? As
> XML file for instance?
>
> As I understoond you are going to list all the dependencies right in
> the build.xml of each module. IMHO, such approach makes not too easy
> to understand the dependencies. I would suppose to have some XML file
> representing kind of dependency table which is accessed by the
> build.xml of each module (by some specific target.
> run.dependants.tests is fine) to retreive the particular list of
> dependencies for the particular module.
> Having all the dependencies in the single place would simplify their
> support. Besides, I expect it would be not too difficult to create
> XSLT to generate HTML table in the perfect human-readable form...
>
> As I said that is just an idea. I'm not experienced in ant-scripting,
> not sure if that is good idea and how it difficult to implement.
>
>
> Wishes,
> --
> Anton Avtamonov,
> Intel Middleware Products Division
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Anton Avtamonov <an...@gmail.com>.
On 4/12/06, George Harley <ge...@googlemail.com> wrote:
> Hi,
>
> Thanks for your comments.
>
> Working on the "walk first then try to run" principle, I will try to
> start adding in "run.dependants.tests" targets to individual modules'
> build.xml scripts over the next few days. Text seems like a good
> candidate to start.
George,
Just a raw idea: it is possible to have description of all
dependencies located in a single place out of individual modules? As
XML file for instance?
As I understoond you are going to list all the dependencies right in
the build.xml of each module. IMHO, such approach makes not too easy
to understand the dependencies. I would suppose to have some XML file
representing kind of dependency table which is accessed by the
build.xml of each module (by some specific target.
run.dependants.tests is fine) to retreive the particular list of
dependencies for the particular module.
Having all the dependencies in the single place would simplify their
support. Besides, I expect it would be not too difficult to create
XSLT to generate HTML table in the perfect human-readable form...
As I said that is just an idea. I'm not experienced in ant-scripting,
not sure if that is good idea and how it difficult to implement.
Wishes,
--
Anton Avtamonov,
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by George Harley <ge...@googlemail.com>.
Hi,
Thanks for your comments.
Working on the "walk first then try to run" principle, I will try to
start adding in "run.dependants.tests" targets to individual modules'
build.xml scripts over the next few days. Text seems like a good
candidate to start.
Best regards,
George
Anton Avtamonov wrote:
> On 4/12/06, Stepan Mishura <st...@gmail.com> wrote:
>
>> Hi George,
>>
>> Your example looks good for me and I think everybody agreed that we should
>> organize testing to avoid running all tests for each update: if you fix bug
>> in 'net' module you don't have to run tests say for 'awt' module but if you
>> update 'luni' then you have to run tests for all modules.
>>
>
> Absolutely agree. Looks like a concept of so-called 'dependency
> matrix'. Its rows and columns represents functional areas. Each cell
> contains a number from 0 to 1 which represents the correlation between
> those functionalilies, diagonal cells (representing self-impact of
> each functional area) obviously contain 1.
> QE usually used that approach to decide which tests needs to be re-executed.
>
> Simplifiing this approach to use only binary dependency (0 or 1) I
> beleive it can be automated so that commiter will just need to call
> some ant target to run all dependent tests. Of course, it needs to be
> kept up to date, however I expect such dependencies are not
> dynamically changed, i.e. it is well-known that almost everything
> depends from luni and almost nothing from net, etc.
>
> --
> Anton Avtamonov,
> Intel Middleware Products Division
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
Anton Avtamonov wrote:
> On 4/12/06, Stepan Mishura <st...@gmail.com> wrote:
>> Hi George,
>>
>> Your example looks good for me and I think everybody agreed that we should
>> organize testing to avoid running all tests for each update: if you fix bug
>> in 'net' module you don't have to run tests say for 'awt' module but if you
>> update 'luni' then you have to run tests for all modules.
>
> Absolutely agree. Looks like a concept of so-called 'dependency
> matrix'. Its rows and columns represents functional areas. Each cell
> contains a number from 0 to 1 which represents the correlation between
> those functionalilies, diagonal cells (representing self-impact of
> each functional area) obviously contain 1.
> QE usually used that approach to decide which tests needs to be re-executed.
>
> Simplifiing this approach to use only binary dependency (0 or 1) I
> beleive it can be automated so that commiter will just need to call
> some ant target to run all dependent tests. Of course, it needs to be
> kept up to date, however I expect such dependencies are not
> dynamically changed, i.e. it is well-known that almost everything
> depends from luni and almost nothing from net, etc.
Would be nice if this allowed one to work their way out of the
dependency graph.
So default is just immediate dependencies. Then some property = 2 means
immediate + the dependencies on the immediate.
Although, I wonder if 2 -> everything anyway....
geir
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Anton Avtamonov <an...@gmail.com>.
On 4/12/06, Stepan Mishura <st...@gmail.com> wrote:
> Hi George,
>
> Your example looks good for me and I think everybody agreed that we should
> organize testing to avoid running all tests for each update: if you fix bug
> in 'net' module you don't have to run tests say for 'awt' module but if you
> update 'luni' then you have to run tests for all modules.
Absolutely agree. Looks like a concept of so-called 'dependency
matrix'. Its rows and columns represents functional areas. Each cell
contains a number from 0 to 1 which represents the correlation between
those functionalilies, diagonal cells (representing self-impact of
each functional area) obviously contain 1.
QE usually used that approach to decide which tests needs to be re-executed.
Simplifiing this approach to use only binary dependency (0 or 1) I
beleive it can be automated so that commiter will just need to call
some ant target to run all dependent tests. Of course, it needs to be
kept up to date, however I expect such dependencies are not
dynamically changed, i.e. it is well-known that almost everything
depends from luni and almost nothing from net, etc.
--
Anton Avtamonov,
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Stepan Mishura <st...@gmail.com>.
Hi George,
Your example looks good for me and I think everybody agreed that we should
organize testing to avoid running all tests for each update: if you fix bug
in 'net' module you don't have to run tests say for 'awt' module but if you
update 'luni' then you have to run tests for all modules.
Thanks,
Stepan.
On 4/11/06, George Harley wrote:
>
> Stepan Mishura wrote:
> > On 4/11/06, Mark Hindess wrote:
> >
> >
> >> Personally, obviously, I'd expect people to run the tests before
> >> committing.
> >>
> >> However, I notice that since enabling the security tests - which fork
> >> for every test - that the tests take over half an hour to run now on
> >> our Linux build machine. So I can see why enthusiasm might lead to
> >> people not running all the tests but instead perhaps just running
> >> those from the module you are changing.
> >>
> >
> >
> > I think that this message[1] relates to the problem with running big
> test
> > suites. So should we do the similar thing i.e. define which test suites
> > should be run for each module?
> >
> > Thanks,
> > Stepan.
> >
> > [1]
> >
> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200603.mbox/%3c442C1C79.1060805@sourcelabs.com%3e
> >
> >
> >
>
> Hi Stepan,
>
> We should both define and then automate which test suites get run for
> changes made to a given module. The dependency information is already
> there for us in the contents of each module's MANIFEST.MF file. So, to
> take a recent example which we are all familiar with ;-) , we can see
> that the java.text package exported from the text module is imported by
> modules logging, LUNI, security and SQL.
>
> Folks wishing to verify that changes made to a given module do not break
> other modules should ideally be able to kick off a JUnit run that
> incorporates all of the tests from the other modules into one test
> suite. This could either be folded into the existing "run.tests" target
> of the module's build.xml or else be a separate target like
> "run.depends.tests" (or similar).
> i.e. something like ...
>
> <target name="run.depends.tests">
>
> <mkdir dir="${hy.tests.reports}" />
>
> <junit fork="yes"
> forkmode="once"
> printsummary="withOutAndErr"
> errorproperty="test.error"
> showoutput="on"
> dir="${hy.text.bin.test}"
> jvm="${hy.target}/jre/bin/java">
>
> <jvmarg value="-showversion"/>
>
> <!-- Required by various tests that set security manager etc -->
> <jvmarg
> value="-
> Djava.security.policy=../../../../support/src/test/resources/config/testing.policy
> "
> />
>
> <!-- Required for running the java.net unit tests -->
> <jvmarg
> value="-
> Dtest.ini.file=../../../../support/src/test/resources/config/localhosttest.ini
> "
> />
>
> <env key="JAVA_HOME" value="${hy.target}/jre"/>
>
> <!-- Adding all dependent test bins to the classpath -->
> <classpath>
> <pathelement path="${hy.text.bin.test}"/>
> <pathelement path="${hy.logging.bin.test}"/>
> <pathelement path="${hy.luni.bin.test}"/>
> <pathelement path="${hy.security.bin.test}"/>
> <pathelement path="${hy.sql.bin.test}"/>
> <pathelement path="${hy.tests.support.bin}" />
> </classpath>
>
> <!-- The DependencyTests suite logically groups the other dependent test
> suites -->
> <test name="tests.text.DependencyTests"
> todir="${hy.tests.reports}"
> haltonfailure="no" >
> <formatter type="xml"/>
> </test>
> </junit>
> </target>
>
>
> In this scenario the class tests.text.DependencyTests would be a simple
> logical grouping of the dependent test suites using standard JUnit
> techniques.
> i.e. something like ...
>
> public class DependencyTests {
> public static Test suite() {
> TestSuite suite = new TestSuite("Dependents on java.text module");
> suite.addTestSuite(org.apache.harmony.logging.AllTests.suite());
> suite.addTestSuite(org.apache.harmony.luni.AllTests.suite());
> suite.addTestSuite(org.apache.harmony.security.AllTests.suite());
> suite.addTestSuite(org.apache.harmony.sql.AllTests.suite());
> suite.addTestSuite(org.apache.harmony.logging.AllTests.suite());
> return suite;
> }
> }
>
> ...and each of the AllTests classes referred to were just a logical
> grouping of the test cases in that particular module.
>
>
> I think that such a scheme will be a tremendous convenience to
> developers as it will increase our confidence that breakages introduced
> into other modules by changes in a particular module will be caught soon
> without recourse to running the entire test suite - just the necessary
> ones.
>
> If we see merit in setting up something like this to run alongside the
> existing test targets then I will be happy to open the JIRA and create
> the new Ant targets and test suites.
>
> Best regards,
> George
>
> >
> >> George just commented in a commit message backing out some changes in
> >> text that all the text tests passed even though the changed caused
> >> failures elsewhere.
> >>
> >> Regards,
> >> Mark.
> >>
> >> On 4/11/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> >>
> >>> Just curious (and this isn't a criticism - I'm just as guilty of not
> >>> doing this)...
> >>>
> >>> Don't you run the tests before committing?
> >>>
> >>> geir
> >>>
> >>> George Harley wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> It *seems* like things started failing after I committed the changes
> >>>>
> >> for
> >>
> >>>> HARMONY-205 last night. I'm looking into this now. If the
> >>>>
> >> investigation
> >>
> >>>> begins to take up too much time I will back the changes out.
> >>>>
> >>>> Best regards,
> >>>> George
> >>>>
> >>>>
> >>>> Stepan Mishura wrote:
> >>>>
> >>>>> The same for me + DatagramChannelTest
> >>>>>
> >>>>> Thanks,
> >>>>> Stepan.
> >>>>>
> >>>>> On 4/11/06, Mark Hindess wrote:
> >>>>>
> >>>>>
> >>>>>> No. These:
> >>>>>>
> >>>>>> F
> >>>>>>
> >>>>>>
> >>
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
> >>
> >>>>>> E
> >>>>>>
> >>>>>>
> >>
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
> >>
> >>>>>> E
> >>>>>>
> >>>>>>
> >>
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
> >>
> >>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
> >>>>>> E
> >>>>>>
> >>>>>>
> >>
> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
> >>
> >>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
> >>>>>> F java.security.cert.X509CertSelectorTest.testMatch
> >>>>>> F java.security.cert.X509CertSelectorTest.testClone
> >>>>>> F tests.api.java.sql.DateTest.testSetTimelong
> >>>>>> F tests.api.java.sql.DateTest.testToString
> >>>>>> F tests.api.java.sql.DateTest.testValueOf
> >>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
> >>>>>> F tests.api.java.sql.TimestampTest.testToString
> >>>>>> F tests.api.java.util.DateTest.test_toGMTString
> >>>>>> F tests.api.java.util.DateTest.test_toString
> >>>>>>
> >>>>>> Regards,
> >>>>>> Mark.
> >>>>>>
> >>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> >>>>>>
> >>>>>>
> >>>>>>> On 4/11/06, Mark Hindess wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get
> >>>>>>>>
> >> this
> >>
> >>>>>>>> property propogated back to the top level ant file was what I was
> >>>>>>>> having trouble with.
> >>>>>>>>
> >>>>>>>> Using a file as you suggest might help. I'll give that a try
> >>>>>>>>
> >>>>>>>>
> >>>>>> shortly...
> >>>>>>
> >>>>>>
> >>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
> >>>>>>>>
> >>>>>>>>
> >>>>>>> I guess that you have 9 tests from DatagramChannelTest passed.
> And
> >>>>>>> 12 +
> >>>>>>>
> >>>>>>>
> >>>>>> 3 =
> >>>>>>
> >>>>>>
> >>>>>>> 15 :-)
> >>>>>>>
> >>>>>>> (And
> >>>>>>>
> >>>>>>>
> >>>>>>>> there are typos - "mathc" should be "match" - in the failure
> >>>>>>>>
> >> messages
> >>
> >>>>>>>> for java.security.cert testMatch and testClone.)
> >>>>>>>>
> >>>>>>>>
> >>>>>>> I've fixed typo.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Stepan.
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>>
> >>>>>>>
> >>>>>>>> Mark.
> >>>>>>>>
> >>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On 4/11/06, Mark Hindess wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Stepan,
> >>>>>>>>>>
> >>>>>>>>>> I have another build running (but without notifications going
> to
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>> the
> >>>>>>
> >>>>>>
> >>>>>>>>>> list) that runs:
> >>>>>>>>>>
> >>>>>>>>>> 1) build (with reference jdk)
> >>>>>>>>>> 2) build with what we created with 1)
> >>>>>>>>>> 3) test
> >>>>>>>>>> 4) create classlists and compare with class load data for
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>> applications
> >>>>>>
> >>>>>>
> >>>>>>>>>> (tomcat, geronimo, continuum, etc.)
> >>>>>>>>>> 5) generate JAPI results
> >>>>>>>>>>
> >>>>>>>>>> I'd like to fail this build at step 3, but I can't see an easy
> >>>>>>>>>>
> >> way
> >>
> >>>>>> to
> >>>>>>
> >>>>>>
> >>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail
> >>>>>>>>>>
> >> if
> >>
> >>>>>> any
> >>>>>>
> >>>>>>
> >>>>>>>>>> of the module test sub-targets had test failures. I could
> parse
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>> the
> >>>>>>
> >>>>>>
> >>>>>>>>>> output I suppose, but I'd rather get ant to propagate the junit
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>> tasks
> >>>>>>
> >>>>>>
> >>>>>>>>>> failure property back up to the top level. I've tried a couple
> >>>>>>>>>>
> >> of
> >>
> >>>>>>>>>> things and none seem to work. Any suggestions welcome.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>> Mark, did you try failureproperty parameter for junit task?
> >>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
> >>>>>>>>>
> >>>>>>>>>
> >>>>>> create
> >>>>>>
> >>>>>>
> >>>>>>>> file "
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail. And
> >>>>>>>>>
> >> in
> >>
> >>>>>> the
> >>>>>>
> >>>>>>
> >>>>>>>> end
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> of tests suite run check whether this file exists on not.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Stepan.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Mark.
> >>>>>>>>>>
> >>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> I've checked out at morning last updates, built the code base
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>> and
> >>>>>>
> >>>>>>
> >>>>>>>> run
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>> the
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> tests …and there are 24 tests failures!
> >>>>>>>>>>>
> >>>>>>>>>>> There are 9 tests failures in
> >>>>>>>>>>>
> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest–
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>> I
> >>>>>>
> >>>>>>
> >>>>>>>> saw
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>> these
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> failures before from time to time. It seems that tests depend
> >>>>>>>>>>>
> >> on
> >>
> >>>>>>>> some
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>> race
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>> these
> >>>>>>
> >>>>>>
> >>>>>>>>>> tests
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> passing for you?
> >>>>>>>>>>>
> >>>>>>>>>>> And there are new 15 test failures. So now if I'll make a
> code
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>> update
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>> or a
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
> >>>>>>>>>>>
> >>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>> notification
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>> with
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>> be
> >>>>>>
> >>>>>>
> >>>>>>>> send
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>> to
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> harmony-commits mailing list. Is it possible to have the same
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>> for
> >>>>>>
> >>>>>>
> >>>>>>>> tests?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>> I
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> mean that after completing automatic build the existing
> >>>>>>>>>>>
> >> classlib
> >>
> >>>>>>>> tests
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>> suite
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> should be run. If there are failing tests then a report
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>> notification
> >>>>>>
> >>>>>>
> >>>>>>>>>> with
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> corresponding subject will be send.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Stepan Mishura
> >>>>>>>>>>> Intel Middleware Products Division
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Mark Hindess <ma...@googlemail.com>
> >>>>>>>>>> IBM Java Technology Centre, UK.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >> ---------------------------------------------------------------------
> >>
> >>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>>>>> To unsubscribe, e-mail:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>> harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>
> >>>>>>
> >>>>>>>>>> For additional commands, e-mail:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>> harmony-dev-help@incubator.apache.org
> >>>>>>
> >>>>>>
> >>>>>>>>> --
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >> ---------------------------------------------------------------------
> >>
> >>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>>>> To unsubscribe, e-mail:
> >>>>>>>>>
> >> harmony-dev-unsubscribe@incubator.apache.org
> >>
> >>>>>>>>> For additional commands, e-mail:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>> harmony-dev-help@incubator.apache.org
> >>>>>>
> >>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Stepan Mishura
> >>>>>>>>> Intel Middleware Products Division
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> --
> >>>>>>>> Mark Hindess <ma...@googlemail.com>
> >>>>>>>> IBM Java Technology Centre, UK.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >> ---------------------------------------------------------------------
> >>
> >>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>>> To unsubscribe, e-mail:
> >>>>>>>>
> >> harmony-dev-unsubscribe@incubator.apache.org
> >>
> >>>>>>>> For additional commands, e-mail:
> >>>>>>>>
> >> harmony-dev-help@incubator.apache.org
> >>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> --
> >>>>>>>
> >>>>>>>
> >> ---------------------------------------------------------------------
> >>
> >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>> To unsubscribe, e-mail:
> >>>>>>>
> >> harmony-dev-unsubscribe@incubator.apache.org
> >>
> >>>>>>> For additional commands, e-mail:
> >>>>>>>
> >> harmony-dev-help@incubator.apache.org
> >>
> >>>>>>> Thanks,
> >>>>>>> Stepan Mishura
> >>>>>>> Intel Middleware Products Division
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> --
> >>>>>> Mark Hindess <ma...@googlemail.com>
> >>>>>> IBM Java Technology Centre, UK.
> >>>>>>
> >>>>>>
> >>>>>>
> >> ---------------------------------------------------------------------
> >>
> >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> >>>>>> For additional commands, e-mail:
> >>>>>>
> >> harmony-dev-help@incubator.apache.org
> >>
> >>>>>>
> >>>>>>
> >>>>> --
> >>>>>
> ---------------------------------------------------------------------
> >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>> For additional commands, e-mail:
> >>>>>
> >> harmony-dev-help@incubator.apache.org
> >>
> >>>>> Thanks,
> >>>>> Stepan Mishura
> >>>>> Intel Middleware Products Division
> >>>>>
> >>>>>
> >>>>>
> >>>> ---------------------------------------------------------------------
> >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> >>>>
> >>>>
> >>>>
> >>> ---------------------------------------------------------------------
> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>
> >>>
> >>>
> >> --
> >> Mark Hindess <ma...@googlemail.com>
> >> IBM Java Technology Centre, UK.
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >>
> >>
> >
> >
> > --
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> > Thanks,
> > Stepan Mishura
> > Intel Middleware Products Division
> >
> >
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Thanks,
Stepan Mishura
Intel Middleware Products Division
Re: classlib test suite status emails?
Posted by George Harley <ge...@googlemail.com>.
Stepan Mishura wrote:
> On 4/11/06, Mark Hindess wrote:
>
>
>> Personally, obviously, I'd expect people to run the tests before
>> committing.
>>
>> However, I notice that since enabling the security tests - which fork
>> for every test - that the tests take over half an hour to run now on
>> our Linux build machine. So I can see why enthusiasm might lead to
>> people not running all the tests but instead perhaps just running
>> those from the module you are changing.
>>
>
>
> I think that this message[1] relates to the problem with running big test
> suites. So should we do the similar thing i.e. define which test suites
> should be run for each module?
>
> Thanks,
> Stepan.
>
> [1]
> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200603.mbox/%3c442C1C79.1060805@sourcelabs.com%3e
>
>
>
Hi Stepan,
We should both define and then automate which test suites get run for
changes made to a given module. The dependency information is already
there for us in the contents of each module's MANIFEST.MF file. So, to
take a recent example which we are all familiar with ;-) , we can see
that the java.text package exported from the text module is imported by
modules logging, LUNI, security and SQL.
Folks wishing to verify that changes made to a given module do not break
other modules should ideally be able to kick off a JUnit run that
incorporates all of the tests from the other modules into one test
suite. This could either be folded into the existing "run.tests" target
of the module's build.xml or else be a separate target like
"run.depends.tests" (or similar).
i.e. something like ...
<target name="run.depends.tests">
<mkdir dir="${hy.tests.reports}" />
<junit fork="yes"
forkmode="once"
printsummary="withOutAndErr"
errorproperty="test.error"
showoutput="on"
dir="${hy.text.bin.test}"
jvm="${hy.target}/jre/bin/java">
<jvmarg value="-showversion"/>
<!-- Required by various tests that set security manager etc -->
<jvmarg
value="-Djava.security.policy=../../../../support/src/test/resources/config/testing.policy"
/>
<!-- Required for running the java.net unit tests -->
<jvmarg
value="-Dtest.ini.file=../../../../support/src/test/resources/config/localhosttest.ini"
/>
<env key="JAVA_HOME" value="${hy.target}/jre"/>
<!-- Adding all dependent test bins to the classpath -->
<classpath>
<pathelement path="${hy.text.bin.test}"/>
<pathelement path="${hy.logging.bin.test}"/>
<pathelement path="${hy.luni.bin.test}"/>
<pathelement path="${hy.security.bin.test}"/>
<pathelement path="${hy.sql.bin.test}"/>
<pathelement path="${hy.tests.support.bin}" />
</classpath>
<!-- The DependencyTests suite logically groups the other dependent test
suites -->
<test name="tests.text.DependencyTests"
todir="${hy.tests.reports}"
haltonfailure="no" >
<formatter type="xml"/>
</test>
</junit>
</target>
In this scenario the class tests.text.DependencyTests would be a simple
logical grouping of the dependent test suites using standard JUnit
techniques.
i.e. something like ...
public class DependencyTests {
public static Test suite() {
TestSuite suite = new TestSuite("Dependents on java.text module");
suite.addTestSuite(org.apache.harmony.logging.AllTests.suite());
suite.addTestSuite(org.apache.harmony.luni.AllTests.suite());
suite.addTestSuite(org.apache.harmony.security.AllTests.suite());
suite.addTestSuite(org.apache.harmony.sql.AllTests.suite());
suite.addTestSuite(org.apache.harmony.logging.AllTests.suite());
return suite;
}
}
...and each of the AllTests classes referred to were just a logical
grouping of the test cases in that particular module.
I think that such a scheme will be a tremendous convenience to
developers as it will increase our confidence that breakages introduced
into other modules by changes in a particular module will be caught soon
without recourse to running the entire test suite - just the necessary
ones.
If we see merit in setting up something like this to run alongside the
existing test targets then I will be happy to open the JIRA and create
the new Ant targets and test suites.
Best regards,
George
>
>> George just commented in a commit message backing out some changes in
>> text that all the text tests passed even though the changed caused
>> failures elsewhere.
>>
>> Regards,
>> Mark.
>>
>> On 4/11/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
>>
>>> Just curious (and this isn't a criticism - I'm just as guilty of not
>>> doing this)...
>>>
>>> Don't you run the tests before committing?
>>>
>>> geir
>>>
>>> George Harley wrote:
>>>
>>>> Hi,
>>>>
>>>> It *seems* like things started failing after I committed the changes
>>>>
>> for
>>
>>>> HARMONY-205 last night. I'm looking into this now. If the
>>>>
>> investigation
>>
>>>> begins to take up too much time I will back the changes out.
>>>>
>>>> Best regards,
>>>> George
>>>>
>>>>
>>>> Stepan Mishura wrote:
>>>>
>>>>> The same for me + DatagramChannelTest
>>>>>
>>>>> Thanks,
>>>>> Stepan.
>>>>>
>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>
>>>>>
>>>>>> No. These:
>>>>>>
>>>>>> F
>>>>>>
>>>>>>
>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>
>>>>>> E
>>>>>>
>>>>>>
>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>
>>>>>> E
>>>>>>
>>>>>>
>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>
>>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>>>> E
>>>>>>
>>>>>>
>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>
>>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>>>> F tests.api.java.sql.DateTest.testToString
>>>>>> F tests.api.java.sql.DateTest.testValueOf
>>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>>>> F tests.api.java.sql.TimestampTest.testToString
>>>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>>>> F tests.api.java.util.DateTest.test_toString
>>>>>>
>>>>>> Regards,
>>>>>> Mark.
>>>>>>
>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>
>>>>>>>
>>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get
>>>>>>>>
>> this
>>
>>>>>>>> property propogated back to the top level ant file was what I was
>>>>>>>> having trouble with.
>>>>>>>>
>>>>>>>> Using a file as you suggest might help. I'll give that a try
>>>>>>>>
>>>>>>>>
>>>>>> shortly...
>>>>>>
>>>>>>
>>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>>>>
>>>>>>>>
>>>>>>> I guess that you have 9 tests from DatagramChannelTest passed. And
>>>>>>> 12 +
>>>>>>>
>>>>>>>
>>>>>> 3 =
>>>>>>
>>>>>>
>>>>>>> 15 :-)
>>>>>>>
>>>>>>> (And
>>>>>>>
>>>>>>>
>>>>>>>> there are typos - "mathc" should be "match" - in the failure
>>>>>>>>
>> messages
>>
>>>>>>>> for java.security.cert testMatch and testClone.)
>>>>>>>>
>>>>>>>>
>>>>>>> I've fixed typo.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Stepan.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>>
>>>>>>>> Mark.
>>>>>>>>
>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Stepan,
>>>>>>>>>>
>>>>>>>>>> I have another build running (but without notifications going to
>>>>>>>>>>
>>>>>>>>>>
>>>>>> the
>>>>>>
>>>>>>
>>>>>>>>>> list) that runs:
>>>>>>>>>>
>>>>>>>>>> 1) build (with reference jdk)
>>>>>>>>>> 2) build with what we created with 1)
>>>>>>>>>> 3) test
>>>>>>>>>> 4) create classlists and compare with class load data for
>>>>>>>>>>
>>>>>>>>>>
>>>>>> applications
>>>>>>
>>>>>>
>>>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>>>> 5) generate JAPI results
>>>>>>>>>>
>>>>>>>>>> I'd like to fail this build at step 3, but I can't see an easy
>>>>>>>>>>
>> way
>>
>>>>>> to
>>>>>>
>>>>>>
>>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail
>>>>>>>>>>
>> if
>>
>>>>>> any
>>>>>>
>>>>>>
>>>>>>>>>> of the module test sub-targets had test failures. I could parse
>>>>>>>>>>
>>>>>>>>>>
>>>>>> the
>>>>>>
>>>>>>
>>>>>>>>>> output I suppose, but I'd rather get ant to propagate the junit
>>>>>>>>>>
>>>>>>>>>>
>>>>>> tasks
>>>>>>
>>>>>>
>>>>>>>>>> failure property back up to the top level. I've tried a couple
>>>>>>>>>>
>> of
>>
>>>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>>>>>>
>>>>>>>>>
>>>>>> create
>>>>>>
>>>>>>
>>>>>>>> file "
>>>>>>>>
>>>>>>>>
>>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail. And
>>>>>>>>>
>> in
>>
>>>>>> the
>>>>>>
>>>>>>
>>>>>>>> end
>>>>>>>>
>>>>>>>>
>>>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Stepan.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Mark.
>>>>>>>>>>
>>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I've checked out at morning last updates, built the code base
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> and
>>>>>>
>>>>>>
>>>>>>>> run
>>>>>>>>
>>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>>>
>>>>>>>>>>> There are 9 tests failures in
>>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest–
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> I
>>>>>>
>>>>>>
>>>>>>>> saw
>>>>>>>>
>>>>>>>>
>>>>>>>>>> these
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> failures before from time to time. It seems that tests depend
>>>>>>>>>>>
>> on
>>
>>>>>>>> some
>>>>>>>>
>>>>>>>>
>>>>>>>>>> race
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> these
>>>>>>
>>>>>>
>>>>>>>>>> tests
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> passing for you?
>>>>>>>>>>>
>>>>>>>>>>> And there are new 15 test failures. So now if I'll make a code
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>> update
>>>>>>>>
>>>>>>>>
>>>>>>>>>> or a
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>>>>>
>>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>> notification
>>>>>>>>
>>>>>>>>
>>>>>>>>>> with
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> be
>>>>>>
>>>>>>
>>>>>>>> send
>>>>>>>>
>>>>>>>>
>>>>>>>>>> to
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> for
>>>>>>
>>>>>>
>>>>>>>> tests?
>>>>>>>>
>>>>>>>>
>>>>>>>>>> I
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> mean that after completing automatic build the existing
>>>>>>>>>>>
>> classlib
>>
>>>>>>>> tests
>>>>>>>>
>>>>>>>>
>>>>>>>>>> suite
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> should be run. If there are failing tests then a report
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> notification
>>>>>>
>>>>>>
>>>>>>>>>> with
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> corresponding subject will be send.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Stepan Mishura
>>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>> ---------------------------------------------------------------------
>>
>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>
>>>>>>>>>>
>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>
>>>>>>
>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>
>>>>>>>>>>
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>> ---------------------------------------------------------------------
>>
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>
>> harmony-dev-unsubscribe@incubator.apache.org
>>
>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>
>>>>>>>>>
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Stepan Mishura
>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>> ---------------------------------------------------------------------
>>
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>
>> harmony-dev-unsubscribe@incubator.apache.org
>>
>>>>>>>> For additional commands, e-mail:
>>>>>>>>
>> harmony-dev-help@incubator.apache.org
>>
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>>
>> ---------------------------------------------------------------------
>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail:
>>>>>>>
>> harmony-dev-unsubscribe@incubator.apache.org
>>
>>>>>>> For additional commands, e-mail:
>>>>>>>
>> harmony-dev-help@incubator.apache.org
>>
>>>>>>> Thanks,
>>>>>>> Stepan Mishura
>>>>>>> Intel Middleware Products Division
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>> IBM Java Technology Centre, UK.
>>>>>>
>>>>>>
>>>>>>
>> ---------------------------------------------------------------------
>>
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>>>>>
>> harmony-dev-help@incubator.apache.org
>>
>>>>>>
>>>>>>
>>>>> --
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail:
>>>>>
>> harmony-dev-help@incubator.apache.org
>>
>>>>> Thanks,
>>>>> Stepan Mishura
>>>>> Intel Middleware Products Division
>>>>>
>>>>>
>>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>>
>> --
>> Mark Hindess <ma...@googlemail.com>
>> IBM Java Technology Centre, UK.
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>>
>
>
> --
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Stepan Mishura <st...@gmail.com>.
On 4/11/06, Mark Hindess wrote:
> Personally, obviously, I'd expect people to run the tests before
> committing.
>
> However, I notice that since enabling the security tests - which fork
> for every test - that the tests take over half an hour to run now on
> our Linux build machine. So I can see why enthusiasm might lead to
> people not running all the tests but instead perhaps just running
> those from the module you are changing.
I think that this message[1] relates to the problem with running big test
suites. So should we do the similar thing i.e. define which test suites
should be run for each module?
Thanks,
Stepan.
[1]
http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200603.mbox/%3c442C1C79.1060805@sourcelabs.com%3e
> George just commented in a commit message backing out some changes in
> text that all the text tests passed even though the changed caused
> failures elsewhere.
>
> Regards,
> Mark.
>
> On 4/11/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> > Just curious (and this isn't a criticism - I'm just as guilty of not
> > doing this)...
> >
> > Don't you run the tests before committing?
> >
> > geir
> >
> > George Harley wrote:
> > > Hi,
> > >
> > > It *seems* like things started failing after I committed the changes
> for
> > > HARMONY-205 last night. I'm looking into this now. If the
> investigation
> > > begins to take up too much time I will back the changes out.
> > >
> > > Best regards,
> > > George
> > >
> > >
> > > Stepan Mishura wrote:
> > >> The same for me + DatagramChannelTest
> > >>
> > >> Thanks,
> > >> Stepan.
> > >>
> > >> On 4/11/06, Mark Hindess wrote:
> > >>
> > >>> No. These:
> > >>>
> > >>> F
> > >>>
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
> > >>>
> > >>> E
> > >>>
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
> > >>>
> > >>> E
> > >>>
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
> > >>>
> > >>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
> > >>> E
> > >>>
> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
> > >>>
> > >>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
> > >>> F java.security.cert.X509CertSelectorTest.testMatch
> > >>> F java.security.cert.X509CertSelectorTest.testClone
> > >>> F tests.api.java.sql.DateTest.testSetTimelong
> > >>> F tests.api.java.sql.DateTest.testToString
> > >>> F tests.api.java.sql.DateTest.testValueOf
> > >>> F tests.api.java.sql.TimestampTest.testSetNanosint
> > >>> F tests.api.java.sql.TimestampTest.testToString
> > >>> F tests.api.java.util.DateTest.test_toGMTString
> > >>> F tests.api.java.util.DateTest.test_toString
> > >>>
> > >>> Regards,
> > >>> Mark.
> > >>>
> > >>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > >>>
> > >>>> On 4/11/06, Mark Hindess wrote:
> > >>>>
> > >>>>> Yes. I was using the failureproperty mechanism. Trying to get
> this
> > >>>>> property propogated back to the top level ant file was what I was
> > >>>>> having trouble with.
> > >>>>>
> > >>>>> Using a file as you suggest might help. I'll give that a try
> > >>>>>
> > >>> shortly...
> > >>>
> > >>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
> > >>>>>
> > >>>> I guess that you have 9 tests from DatagramChannelTest passed. And
> > >>>> 12 +
> > >>>>
> > >>> 3 =
> > >>>
> > >>>> 15 :-)
> > >>>>
> > >>>> (And
> > >>>>
> > >>>>> there are typos - "mathc" should be "match" - in the failure
> messages
> > >>>>> for java.security.cert testMatch and testClone.)
> > >>>>>
> > >>>> I've fixed typo.
> > >>>>
> > >>>> Thanks,
> > >>>> Stepan.
> > >>>>
> > >>>> Regards,
> > >>>>
> > >>>>> Mark.
> > >>>>>
> > >>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > >>>>>
> > >>>>>> On 4/11/06, Mark Hindess wrote:
> > >>>>>>
> > >>>>>>
> > >>>>>>> Stepan,
> > >>>>>>>
> > >>>>>>> I have another build running (but without notifications going to
> > >>>>>>>
> > >>> the
> > >>>
> > >>>>>>> list) that runs:
> > >>>>>>>
> > >>>>>>> 1) build (with reference jdk)
> > >>>>>>> 2) build with what we created with 1)
> > >>>>>>> 3) test
> > >>>>>>> 4) create classlists and compare with class load data for
> > >>>>>>>
> > >>> applications
> > >>>
> > >>>>>>> (tomcat, geronimo, continuum, etc.)
> > >>>>>>> 5) generate JAPI results
> > >>>>>>>
> > >>>>>>> I'd like to fail this build at step 3, but I can't see an easy
> way
> > >>>>>>>
> > >>> to
> > >>>
> > >>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail
> if
> > >>>>>>>
> > >>> any
> > >>>
> > >>>>>>> of the module test sub-targets had test failures. I could parse
> > >>>>>>>
> > >>> the
> > >>>
> > >>>>>>> output I suppose, but I'd rather get ant to propagate the junit
> > >>>>>>>
> > >>> tasks
> > >>>
> > >>>>>>> failure property back up to the top level. I've tried a couple
> of
> > >>>>>>> things and none seem to work. Any suggestions welcome.
> > >>>>>>>
> > >>>>>> Mark, did you try failureproperty parameter for junit task?
> > >>>>>> We may add it to ant sub-targets to raise a flag, for example,
> > >>>>>>
> > >>> create
> > >>>
> > >>>>> file "
> > >>>>>
> > >>>>>> TESTS.FAILED" in the root, when tests for some module fail. And
> in
> > >>>>>>
> > >>> the
> > >>>
> > >>>>> end
> > >>>>>
> > >>>>>> of tests suite run check whether this file exists on not.
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> Stepan.
> > >>>>>>
> > >>>>>>
> > >>>>>> Regards,
> > >>>>>>
> > >>>>>>> Mark.
> > >>>>>>>
> > >>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> Hi,
> > >>>>>>>>
> > >>>>>>>> I've checked out at morning last updates, built the code base
> > >>>>>>>>
> > >>> and
> > >>>
> > >>>>> run
> > >>>>>
> > >>>>>>> the
> > >>>>>>>
> > >>>>>>>> tests …and there are 24 tests failures!
> > >>>>>>>>
> > >>>>>>>> There are 9 tests failures in
> > >>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest–
> > >>>>>>>>
> > >>> I
> > >>>
> > >>>>> saw
> > >>>>>
> > >>>>>>> these
> > >>>>>>>
> > >>>>>>>> failures before from time to time. It seems that tests depend
> on
> > >>>>>>>>
> > >>>>> some
> > >>>>>
> > >>>>>>> race
> > >>>>>>>
> > >>>>>>>> conditions because they may pass if I rerun them. Paulex, are
> > >>>>>>>>
> > >>> these
> > >>>
> > >>>>>>> tests
> > >>>>>>>
> > >>>>>>>> passing for you?
> > >>>>>>>>
> > >>>>>>>> And there are new 15 test failures. So now if I'll make a code
> > >>>>>>>>
> > >>>>> update
> > >>>>>
> > >>>>>>> or a
> > >>>>>>>
> > >>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
> > >>>>>>>>
> > >>>>>>>> Currently if a commit breaks the Harmony classlib build a
> > >>>>>>>>
> > >>>>> notification
> > >>>>>
> > >>>>>>> with
> > >>>>>>>
> > >>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
> > >>>>>>>>
> > >>> be
> > >>>
> > >>>>> send
> > >>>>>
> > >>>>>>> to
> > >>>>>>>
> > >>>>>>>> harmony-commits mailing list. Is it possible to have the same
> > >>>>>>>>
> > >>> for
> > >>>
> > >>>>> tests?
> > >>>>>
> > >>>>>>> I
> > >>>>>>>
> > >>>>>>>> mean that after completing automatic build the existing
> classlib
> > >>>>>>>>
> > >>>>> tests
> > >>>>>
> > >>>>>>> suite
> > >>>>>>>
> > >>>>>>>> should be run. If there are failing tests then a report
> > >>>>>>>>
> > >>> notification
> > >>>
> > >>>>>>> with
> > >>>>>>>
> > >>>>>>>> corresponding subject will be send.
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> Stepan Mishura
> > >>>>>>>> Intel Middleware Products Division
> > >>>>>>>>
> > >>>>>>> --
> > >>>>>>> Mark Hindess <ma...@googlemail.com>
> > >>>>>>> IBM Java Technology Centre, UK.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>
> ---------------------------------------------------------------------
> > >>>
> > >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>>>>> To unsubscribe, e-mail:
> > >>>>>>>
> > >>> harmony-dev-unsubscribe@incubator.apache.org
> > >>>
> > >>>>>>> For additional commands, e-mail:
> > >>>>>>>
> > >>> harmony-dev-help@incubator.apache.org
> > >>>
> > >>>>>>>
> > >>>>>> --
> > >>>>>>
> > >>>>>>
> > >>>
> ---------------------------------------------------------------------
> > >>>
> > >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>>>> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > >>>>>> For additional commands, e-mail:
> > >>>>>>
> > >>> harmony-dev-help@incubator.apache.org
> > >>>
> > >>>>>> Thanks,
> > >>>>>> Stepan Mishura
> > >>>>>> Intel Middleware Products Division
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>> --
> > >>>>> Mark Hindess <ma...@googlemail.com>
> > >>>>> IBM Java Technology Centre, UK.
> > >>>>>
> > >>>>>
> ---------------------------------------------------------------------
> > >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>>> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > >>>>> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>> --
> > >>>>
> ---------------------------------------------------------------------
> > >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > >>>> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > >>>>
> > >>>> Thanks,
> > >>>> Stepan Mishura
> > >>>> Intel Middleware Products Division
> > >>>>
> > >>>>
> > >>>>
> > >>> --
> > >>> Mark Hindess <ma...@googlemail.com>
> > >>> IBM Java Technology Centre, UK.
> > >>>
> > >>>
> ---------------------------------------------------------------------
> > >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >>> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > >>>
> > >>>
> > >>>
> > >>
> > >>
> > >> --
> > >> ---------------------------------------------------------------------
> > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > >>
> > >> Thanks,
> > >> Stepan Mishura
> > >> Intel Middleware Products Division
> > >>
> > >>
> > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> --
> Mark Hindess <ma...@googlemail.com>
> IBM Java Technology Centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Thanks,
Stepan Mishura
Intel Middleware Products Division
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
DNS timeouts are usually quite high ... of the order of 30-60 seconds
so a number of these could possibly add up to 10 minutes. Given the
meaningful test name (I wont join that discussion yet!) then this
seems like quite a likely source of problems.
Regards,
Mark.
On 4/14/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> what could it possibly be doing for 10 minutes?
>
> Stepan Mishura wrote:
> > Mikhail, you missed the champion!
> > I've just look to the generated test report: total time is 1143.767 and time
> > for
> > tests.api.java.net.Inet6AddressTest.test_getByNameLjava_lang_String is
> > 645.167!!!
> >
> > I'd exclude this test to speed up test suite run in 2 times.
> >
> > Thanks,
> > Stepan.
> >
> > On 4/12/06, Mikhail Loenko wrote:
> >> Top 12 winners out of 758 tests (1.6%) according to junit report
> >> take 458 sec out of 572 (80%)
> >>
> >> The winners are:
> >>
> >> IdentityScopeTest 112.031
> >> IdentityTest 85.233
> >> Inet6AddressTest 57.944
> >> NTLoginModuleTest 55.199
> >> OutOfMemoryErrorTest 30.724
> >> SignatureTest 26.859
> >> ObjectTest 19.148
> >> ThreadGroupTest 18.878
> >> DatagramChannelTest 14.852
> >> SocketChannelTest 14.171
> >> TimerTest 12.408
> >> KeyPairGenerator3Test 10.405
> >>
> >> Thanks,
> >> Mikhail
> >>
> >> 2006/4/12, Mikhail Loenko <ml...@gmail.com>:
> >>> I've fixed tests that change security manager and set of providers,
> >>> now the fork mode for the security tests is "once", run time reduced
> >>> by ~7 minutes.
> >>>
> >>> So now there are other champions, should we go through them and
> >>> reduce/exclude/move to another suite?
> >>>
> >>> Thanks,
> >>> Mikhail
> >>>
> >>> 2006/4/11, Mark Hindess <ma...@googlemail.com>:
> >>>> Personally, obviously, I'd expect people to run the tests before
> >> committing.
> >>>> However, I notice that since enabling the security tests - which fork
> >>>> for every test - that the tests take over half an hour to run now on
> >>>> our Linux build machine. So I can see why enthusiasm might lead to
> >>>> people not running all the tests but instead perhaps just running
> >>>> those from the module you are changing.
> >>>>
> >>>> George just commented in a commit message backing out some changes in
> >>>> text that all the text tests passed even though the changed caused
> >>>> failures elsewhere.
> >>>>
> >>>> Regards,
> >>>> Mark.
> >
> > [SNIP]
> >
> > -----------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> > Thanks,
> > Stepan Mishura
> > Intel Middleware Products Division
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
what could it possibly be doing for 10 minutes?
Stepan Mishura wrote:
> Mikhail, you missed the champion!
> I've just look to the generated test report: total time is 1143.767 and time
> for
> tests.api.java.net.Inet6AddressTest.test_getByNameLjava_lang_String is
> 645.167!!!
>
> I'd exclude this test to speed up test suite run in 2 times.
>
> Thanks,
> Stepan.
>
> On 4/12/06, Mikhail Loenko wrote:
>> Top 12 winners out of 758 tests (1.6%) according to junit report
>> take 458 sec out of 572 (80%)
>>
>> The winners are:
>>
>> IdentityScopeTest 112.031
>> IdentityTest 85.233
>> Inet6AddressTest 57.944
>> NTLoginModuleTest 55.199
>> OutOfMemoryErrorTest 30.724
>> SignatureTest 26.859
>> ObjectTest 19.148
>> ThreadGroupTest 18.878
>> DatagramChannelTest 14.852
>> SocketChannelTest 14.171
>> TimerTest 12.408
>> KeyPairGenerator3Test 10.405
>>
>> Thanks,
>> Mikhail
>>
>> 2006/4/12, Mikhail Loenko <ml...@gmail.com>:
>>> I've fixed tests that change security manager and set of providers,
>>> now the fork mode for the security tests is "once", run time reduced
>>> by ~7 minutes.
>>>
>>> So now there are other champions, should we go through them and
>>> reduce/exclude/move to another suite?
>>>
>>> Thanks,
>>> Mikhail
>>>
>>> 2006/4/11, Mark Hindess <ma...@googlemail.com>:
>>>> Personally, obviously, I'd expect people to run the tests before
>> committing.
>>>> However, I notice that since enabling the security tests - which fork
>>>> for every test - that the tests take over half an hour to run now on
>>>> our Linux build machine. So I can see why enthusiasm might lead to
>>>> people not running all the tests but instead perhaps just running
>>>> those from the module you are changing.
>>>>
>>>> George just commented in a commit message backing out some changes in
>>>> text that all the text tests passed even though the changed caused
>>>> failures elsewhere.
>>>>
>>>> Regards,
>>>> Mark.
>
> [SNIP]
>
> -----------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Stepan Mishura <st...@gmail.com>.
Interesting this true for Linux but not for Windows ...
Thanks,
Stepan
On 4/14/06, Stepan Mishura wrote:
>
> Mikhail, you missed the champion!
> I've just look to the generated test report: total time is 1143.767 and
> time for
> tests.api.java.net.Inet6AddressTest.test_getByNameLjava_lang_String is
> 645.167!!!
>
> I'd exclude this test to speed up test suite run in 2 times.
>
> Thanks,
> Stepan.
>
> On 4/12/06, Mikhail Loenko wrote:
>
> > Top 12 winners out of 758 tests (1.6%) according to junit report
> > take 458 sec out of 572 (80%)
> >
> > The winners are:
> >
> > IdentityScopeTest 112.031
> > IdentityTest 85.233
> > Inet6AddressTest 57.944
> > NTLoginModuleTest 55.199
> > OutOfMemoryErrorTest 30.724
> > SignatureTest 26.859
> > ObjectTest 19.148
> > ThreadGroupTest 18.878
> > DatagramChannelTest 14.852
> > SocketChannelTest 14.171
> > TimerTest 12.408
> > KeyPairGenerator3Test 10.405
> >
> > Thanks,
> > Mikhail
> >
> > 2006/4/12, Mikhail Loenko < mloenko@gmail.com>:
> > > I've fixed tests that change security manager and set of providers,
> > > now the fork mode for the security tests is "once", run time reduced
> > > by ~7 minutes.
> > >
> > > So now there are other champions, should we go through them and
> > > reduce/exclude/move to another suite?
> > >
> > > Thanks,
> > > Mikhail
> > >
> > > 2006/4/11, Mark Hindess < mark.hindess@googlemail.com>:
> > > > Personally, obviously, I'd expect people to run the tests before
> > committing.
> > > >
> > > > However, I notice that since enabling the security tests - which
> > fork
> > > > for every test - that the tests take over half an hour to run now on
> > > > our Linux build machine. So I can see why enthusiasm might lead to
> > > > people not running all the tests but instead perhaps just running
> > > > those from the module you are changing.
> > > >
> > > > George just commented in a commit message backing out some changes
> > in
> > > > text that all the text tests passed even though the changed caused
> > > > failures elsewhere.
> > > >
> > > > Regards,
> > > > Mark.
>
> [SNIP]
>
> -----------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
--
-----------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Thanks,
Stepan Mishura
Intel Middleware Products Division
Re: classlib test suite status emails?
Posted by Stepan Mishura <st...@gmail.com>.
Mikhail, you missed the champion!
I've just look to the generated test report: total time is 1143.767 and time
for
tests.api.java.net.Inet6AddressTest.test_getByNameLjava_lang_String is
645.167!!!
I'd exclude this test to speed up test suite run in 2 times.
Thanks,
Stepan.
On 4/12/06, Mikhail Loenko wrote:
>
> Top 12 winners out of 758 tests (1.6%) according to junit report
> take 458 sec out of 572 (80%)
>
> The winners are:
>
> IdentityScopeTest 112.031
> IdentityTest 85.233
> Inet6AddressTest 57.944
> NTLoginModuleTest 55.199
> OutOfMemoryErrorTest 30.724
> SignatureTest 26.859
> ObjectTest 19.148
> ThreadGroupTest 18.878
> DatagramChannelTest 14.852
> SocketChannelTest 14.171
> TimerTest 12.408
> KeyPairGenerator3Test 10.405
>
> Thanks,
> Mikhail
>
> 2006/4/12, Mikhail Loenko <ml...@gmail.com>:
> > I've fixed tests that change security manager and set of providers,
> > now the fork mode for the security tests is "once", run time reduced
> > by ~7 minutes.
> >
> > So now there are other champions, should we go through them and
> > reduce/exclude/move to another suite?
> >
> > Thanks,
> > Mikhail
> >
> > 2006/4/11, Mark Hindess <ma...@googlemail.com>:
> > > Personally, obviously, I'd expect people to run the tests before
> committing.
> > >
> > > However, I notice that since enabling the security tests - which fork
> > > for every test - that the tests take over half an hour to run now on
> > > our Linux build machine. So I can see why enthusiasm might lead to
> > > people not running all the tests but instead perhaps just running
> > > those from the module you are changing.
> > >
> > > George just commented in a commit message backing out some changes in
> > > text that all the text tests passed even though the changed caused
> > > failures elsewhere.
> > >
> > > Regards,
> > > Mark.
[SNIP]
-----------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Thanks,
Stepan Mishura
Intel Middleware Products Division
Re: classlib test suite status emails?
Posted by Mikhail Loenko <ml...@gmail.com>.
Top 12 winners out of 758 tests (1.6%) according to junit report
take 458 sec out of 572 (80%)
The winners are:
IdentityScopeTest 112.031
IdentityTest 85.233
Inet6AddressTest 57.944
NTLoginModuleTest 55.199
OutOfMemoryErrorTest 30.724
SignatureTest 26.859
ObjectTest 19.148
ThreadGroupTest 18.878
DatagramChannelTest 14.852
SocketChannelTest 14.171
TimerTest 12.408
KeyPairGenerator3Test 10.405
Thanks,
Mikhail
2006/4/12, Mikhail Loenko <ml...@gmail.com>:
> I've fixed tests that change security manager and set of providers,
> now the fork mode for the security tests is "once", run time reduced
> by ~7 minutes.
>
> So now there are other champions, should we go through them and
> reduce/exclude/move to another suite?
>
> Thanks,
> Mikhail
>
> 2006/4/11, Mark Hindess <ma...@googlemail.com>:
> > Personally, obviously, I'd expect people to run the tests before committing.
> >
> > However, I notice that since enabling the security tests - which fork
> > for every test - that the tests take over half an hour to run now on
> > our Linux build machine. So I can see why enthusiasm might lead to
> > people not running all the tests but instead perhaps just running
> > those from the module you are changing.
> >
> > George just commented in a commit message backing out some changes in
> > text that all the text tests passed even though the changed caused
> > failures elsewhere.
> >
> > Regards,
> > Mark.
> >
> > On 4/11/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> > > Just curious (and this isn't a criticism - I'm just as guilty of not
> > > doing this)...
> > >
> > > Don't you run the tests before committing?
> > >
> > > geir
> > >
> > > George Harley wrote:
> > > > Hi,
> > > >
> > > > It *seems* like things started failing after I committed the changes for
> > > > HARMONY-205 last night. I'm looking into this now. If the investigation
> > > > begins to take up too much time I will back the changes out.
> > > >
> > > > Best regards,
> > > > George
> > > >
> > > >
> > > > Stepan Mishura wrote:
> > > >> The same for me + DatagramChannelTest
> > > >>
> > > >> Thanks,
> > > >> Stepan.
> > > >>
> > > >> On 4/11/06, Mark Hindess wrote:
> > > >>
> > > >>> No. These:
> > > >>>
> > > >>> F
> > > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
> > > >>>
> > > >>> E
> > > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
> > > >>>
> > > >>> E
> > > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
> > > >>>
> > > >>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
> > > >>> E
> > > >>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
> > > >>>
> > > >>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
> > > >>> F java.security.cert.X509CertSelectorTest.testMatch
> > > >>> F java.security.cert.X509CertSelectorTest.testClone
> > > >>> F tests.api.java.sql.DateTest.testSetTimelong
> > > >>> F tests.api.java.sql.DateTest.testToString
> > > >>> F tests.api.java.sql.DateTest.testValueOf
> > > >>> F tests.api.java.sql.TimestampTest.testSetNanosint
> > > >>> F tests.api.java.sql.TimestampTest.testToString
> > > >>> F tests.api.java.util.DateTest.test_toGMTString
> > > >>> F tests.api.java.util.DateTest.test_toString
> > > >>>
> > > >>> Regards,
> > > >>> Mark.
> > > >>>
> > > >>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > >>>
> > > >>>> On 4/11/06, Mark Hindess wrote:
> > > >>>>
> > > >>>>> Yes. I was using the failureproperty mechanism. Trying to get this
> > > >>>>> property propogated back to the top level ant file was what I was
> > > >>>>> having trouble with.
> > > >>>>>
> > > >>>>> Using a file as you suggest might help. I'll give that a try
> > > >>>>>
> > > >>> shortly...
> > > >>>
> > > >>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
> > > >>>>>
> > > >>>> I guess that you have 9 tests from DatagramChannelTest passed. And
> > > >>>> 12 +
> > > >>>>
> > > >>> 3 =
> > > >>>
> > > >>>> 15 :-)
> > > >>>>
> > > >>>> (And
> > > >>>>
> > > >>>>> there are typos - "mathc" should be "match" - in the failure messages
> > > >>>>> for java.security.cert testMatch and testClone.)
> > > >>>>>
> > > >>>> I've fixed typo.
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Stepan.
> > > >>>>
> > > >>>> Regards,
> > > >>>>
> > > >>>>> Mark.
> > > >>>>>
> > > >>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > >>>>>
> > > >>>>>> On 4/11/06, Mark Hindess wrote:
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>> Stepan,
> > > >>>>>>>
> > > >>>>>>> I have another build running (but without notifications going to
> > > >>>>>>>
> > > >>> the
> > > >>>
> > > >>>>>>> list) that runs:
> > > >>>>>>>
> > > >>>>>>> 1) build (with reference jdk)
> > > >>>>>>> 2) build with what we created with 1)
> > > >>>>>>> 3) test
> > > >>>>>>> 4) create classlists and compare with class load data for
> > > >>>>>>>
> > > >>> applications
> > > >>>
> > > >>>>>>> (tomcat, geronimo, continuum, etc.)
> > > >>>>>>> 5) generate JAPI results
> > > >>>>>>>
> > > >>>>>>> I'd like to fail this build at step 3, but I can't see an easy way
> > > >>>>>>>
> > > >>> to
> > > >>>
> > > >>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail if
> > > >>>>>>>
> > > >>> any
> > > >>>
> > > >>>>>>> of the module test sub-targets had test failures. I could parse
> > > >>>>>>>
> > > >>> the
> > > >>>
> > > >>>>>>> output I suppose, but I'd rather get ant to propagate the junit
> > > >>>>>>>
> > > >>> tasks
> > > >>>
> > > >>>>>>> failure property back up to the top level. I've tried a couple of
> > > >>>>>>> things and none seem to work. Any suggestions welcome.
> > > >>>>>>>
> > > >>>>>> Mark, did you try failureproperty parameter for junit task?
> > > >>>>>> We may add it to ant sub-targets to raise a flag, for example,
> > > >>>>>>
> > > >>> create
> > > >>>
> > > >>>>> file "
> > > >>>>>
> > > >>>>>> TESTS.FAILED" in the root, when tests for some module fail. And in
> > > >>>>>>
> > > >>> the
> > > >>>
> > > >>>>> end
> > > >>>>>
> > > >>>>>> of tests suite run check whether this file exists on not.
> > > >>>>>>
> > > >>>>>> Thanks,
> > > >>>>>> Stepan.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Regards,
> > > >>>>>>
> > > >>>>>>> Mark.
> > > >>>>>>>
> > > >>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > >>>>>>>
> > > >>>>>>>> Hi,
> > > >>>>>>>>
> > > >>>>>>>> I've checked out at morning last updates, built the code base
> > > >>>>>>>>
> > > >>> and
> > > >>>
> > > >>>>> run
> > > >>>>>
> > > >>>>>>> the
> > > >>>>>>>
> > > >>>>>>>> tests …and there are 24 tests failures!
> > > >>>>>>>>
> > > >>>>>>>> There are 9 tests failures in
> > > >>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest –
> > > >>>>>>>>
> > > >>> I
> > > >>>
> > > >>>>> saw
> > > >>>>>
> > > >>>>>>> these
> > > >>>>>>>
> > > >>>>>>>> failures before from time to time. It seems that tests depend on
> > > >>>>>>>>
> > > >>>>> some
> > > >>>>>
> > > >>>>>>> race
> > > >>>>>>>
> > > >>>>>>>> conditions because they may pass if I rerun them. Paulex, are
> > > >>>>>>>>
> > > >>> these
> > > >>>
> > > >>>>>>> tests
> > > >>>>>>>
> > > >>>>>>>> passing for you?
> > > >>>>>>>>
> > > >>>>>>>> And there are new 15 test failures. So now if I'll make a code
> > > >>>>>>>>
> > > >>>>> update
> > > >>>>>
> > > >>>>>>> or a
> > > >>>>>>>
> > > >>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
> > > >>>>>>>>
> > > >>>>>>>> Currently if a commit breaks the Harmony classlib build a
> > > >>>>>>>>
> > > >>>>> notification
> > > >>>>>
> > > >>>>>>> with
> > > >>>>>>>
> > > >>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
> > > >>>>>>>>
> > > >>> be
> > > >>>
> > > >>>>> send
> > > >>>>>
> > > >>>>>>> to
> > > >>>>>>>
> > > >>>>>>>> harmony-commits mailing list. Is it possible to have the same
> > > >>>>>>>>
> > > >>> for
> > > >>>
> > > >>>>> tests?
> > > >>>>>
> > > >>>>>>> I
> > > >>>>>>>
> > > >>>>>>>> mean that after completing automatic build the existing classlib
> > > >>>>>>>>
> > > >>>>> tests
> > > >>>>>
> > > >>>>>>> suite
> > > >>>>>>>
> > > >>>>>>>> should be run. If there are failing tests then a report
> > > >>>>>>>>
> > > >>> notification
> > > >>>
> > > >>>>>>> with
> > > >>>>>>>
> > > >>>>>>>> corresponding subject will be send.
> > > >>>>>>>>
> > > >>>>>>>> Thanks,
> > > >>>>>>>> Stepan Mishura
> > > >>>>>>>> Intel Middleware Products Division
> > > >>>>>>>>
> > > >>>>>>> --
> > > >>>>>>> Mark Hindess <ma...@googlemail.com>
> > > >>>>>>> IBM Java Technology Centre, UK.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>> ---------------------------------------------------------------------
> > > >>>
> > > >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >>>>>>> To unsubscribe, e-mail:
> > > >>>>>>>
> > > >>> harmony-dev-unsubscribe@incubator.apache.org
> > > >>>
> > > >>>>>>> For additional commands, e-mail:
> > > >>>>>>>
> > > >>> harmony-dev-help@incubator.apache.org
> > > >>>
> > > >>>>>>>
> > > >>>>>> --
> > > >>>>>>
> > > >>>>>>
> > > >>> ---------------------------------------------------------------------
> > > >>>
> > > >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > >>>>>> For additional commands, e-mail:
> > > >>>>>>
> > > >>> harmony-dev-help@incubator.apache.org
> > > >>>
> > > >>>>>> Thanks,
> > > >>>>>> Stepan Mishura
> > > >>>>>> Intel Middleware Products Division
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>> --
> > > >>>>> Mark Hindess <ma...@googlemail.com>
> > > >>>>> IBM Java Technology Centre, UK.
> > > >>>>>
> > > >>>>> ---------------------------------------------------------------------
> > > >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > >>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>> --
> > > >>>> ---------------------------------------------------------------------
> > > >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Stepan Mishura
> > > >>>> Intel Middleware Products Division
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>> --
> > > >>> Mark Hindess <ma...@googlemail.com>
> > > >>> IBM Java Technology Centre, UK.
> > > >>>
> > > >>> ---------------------------------------------------------------------
> > > >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >> --
> > > >> ---------------------------------------------------------------------
> > > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >>
> > > >> Thanks,
> > > >> Stepan Mishura
> > > >> Intel Middleware Products Division
> > > >>
> > > >>
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
> > --
> > Mark Hindess <ma...@googlemail.com>
> > IBM Java Technology Centre, UK.
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
On 4/12/06, Mikhail Loenko <ml...@gmail.com> wrote:
<
> I've fixed tests that change security manager and set of providers,
> now the fork mode for the security tests is "once", run time reduced
> by ~7 minutes.
Yippee! Thanks Mikhail. This makes testing much easier.
Regards,
Mark.
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Mikhail Loenko <ml...@gmail.com>.
I've fixed tests that change security manager and set of providers,
now the fork mode for the security tests is "once", run time reduced
by ~7 minutes.
So now there are other champions, should we go through them and
reduce/exclude/move to another suite?
Thanks,
Mikhail
2006/4/11, Mark Hindess <ma...@googlemail.com>:
> Personally, obviously, I'd expect people to run the tests before committing.
>
> However, I notice that since enabling the security tests - which fork
> for every test - that the tests take over half an hour to run now on
> our Linux build machine. So I can see why enthusiasm might lead to
> people not running all the tests but instead perhaps just running
> those from the module you are changing.
>
> George just commented in a commit message backing out some changes in
> text that all the text tests passed even though the changed caused
> failures elsewhere.
>
> Regards,
> Mark.
>
> On 4/11/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> > Just curious (and this isn't a criticism - I'm just as guilty of not
> > doing this)...
> >
> > Don't you run the tests before committing?
> >
> > geir
> >
> > George Harley wrote:
> > > Hi,
> > >
> > > It *seems* like things started failing after I committed the changes for
> > > HARMONY-205 last night. I'm looking into this now. If the investigation
> > > begins to take up too much time I will back the changes out.
> > >
> > > Best regards,
> > > George
> > >
> > >
> > > Stepan Mishura wrote:
> > >> The same for me + DatagramChannelTest
> > >>
> > >> Thanks,
> > >> Stepan.
> > >>
> > >> On 4/11/06, Mark Hindess wrote:
> > >>
> > >>> No. These:
> > >>>
> > >>> F
> > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
> > >>>
> > >>> E
> > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
> > >>>
> > >>> E
> > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
> > >>>
> > >>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
> > >>> E
> > >>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
> > >>>
> > >>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
> > >>> F java.security.cert.X509CertSelectorTest.testMatch
> > >>> F java.security.cert.X509CertSelectorTest.testClone
> > >>> F tests.api.java.sql.DateTest.testSetTimelong
> > >>> F tests.api.java.sql.DateTest.testToString
> > >>> F tests.api.java.sql.DateTest.testValueOf
> > >>> F tests.api.java.sql.TimestampTest.testSetNanosint
> > >>> F tests.api.java.sql.TimestampTest.testToString
> > >>> F tests.api.java.util.DateTest.test_toGMTString
> > >>> F tests.api.java.util.DateTest.test_toString
> > >>>
> > >>> Regards,
> > >>> Mark.
> > >>>
> > >>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > >>>
> > >>>> On 4/11/06, Mark Hindess wrote:
> > >>>>
> > >>>>> Yes. I was using the failureproperty mechanism. Trying to get this
> > >>>>> property propogated back to the top level ant file was what I was
> > >>>>> having trouble with.
> > >>>>>
> > >>>>> Using a file as you suggest might help. I'll give that a try
> > >>>>>
> > >>> shortly...
> > >>>
> > >>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
> > >>>>>
> > >>>> I guess that you have 9 tests from DatagramChannelTest passed. And
> > >>>> 12 +
> > >>>>
> > >>> 3 =
> > >>>
> > >>>> 15 :-)
> > >>>>
> > >>>> (And
> > >>>>
> > >>>>> there are typos - "mathc" should be "match" - in the failure messages
> > >>>>> for java.security.cert testMatch and testClone.)
> > >>>>>
> > >>>> I've fixed typo.
> > >>>>
> > >>>> Thanks,
> > >>>> Stepan.
> > >>>>
> > >>>> Regards,
> > >>>>
> > >>>>> Mark.
> > >>>>>
> > >>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > >>>>>
> > >>>>>> On 4/11/06, Mark Hindess wrote:
> > >>>>>>
> > >>>>>>
> > >>>>>>> Stepan,
> > >>>>>>>
> > >>>>>>> I have another build running (but without notifications going to
> > >>>>>>>
> > >>> the
> > >>>
> > >>>>>>> list) that runs:
> > >>>>>>>
> > >>>>>>> 1) build (with reference jdk)
> > >>>>>>> 2) build with what we created with 1)
> > >>>>>>> 3) test
> > >>>>>>> 4) create classlists and compare with class load data for
> > >>>>>>>
> > >>> applications
> > >>>
> > >>>>>>> (tomcat, geronimo, continuum, etc.)
> > >>>>>>> 5) generate JAPI results
> > >>>>>>>
> > >>>>>>> I'd like to fail this build at step 3, but I can't see an easy way
> > >>>>>>>
> > >>> to
> > >>>
> > >>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail if
> > >>>>>>>
> > >>> any
> > >>>
> > >>>>>>> of the module test sub-targets had test failures. I could parse
> > >>>>>>>
> > >>> the
> > >>>
> > >>>>>>> output I suppose, but I'd rather get ant to propagate the junit
> > >>>>>>>
> > >>> tasks
> > >>>
> > >>>>>>> failure property back up to the top level. I've tried a couple of
> > >>>>>>> things and none seem to work. Any suggestions welcome.
> > >>>>>>>
> > >>>>>> Mark, did you try failureproperty parameter for junit task?
> > >>>>>> We may add it to ant sub-targets to raise a flag, for example,
> > >>>>>>
> > >>> create
> > >>>
> > >>>>> file "
> > >>>>>
> > >>>>>> TESTS.FAILED" in the root, when tests for some module fail. And in
> > >>>>>>
> > >>> the
> > >>>
> > >>>>> end
> > >>>>>
> > >>>>>> of tests suite run check whether this file exists on not.
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> Stepan.
> > >>>>>>
> > >>>>>>
> > >>>>>> Regards,
> > >>>>>>
> > >>>>>>> Mark.
> > >>>>>>>
> > >>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> Hi,
> > >>>>>>>>
> > >>>>>>>> I've checked out at morning last updates, built the code base
> > >>>>>>>>
> > >>> and
> > >>>
> > >>>>> run
> > >>>>>
> > >>>>>>> the
> > >>>>>>>
> > >>>>>>>> tests …and there are 24 tests failures!
> > >>>>>>>>
> > >>>>>>>> There are 9 tests failures in
> > >>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest –
> > >>>>>>>>
> > >>> I
> > >>>
> > >>>>> saw
> > >>>>>
> > >>>>>>> these
> > >>>>>>>
> > >>>>>>>> failures before from time to time. It seems that tests depend on
> > >>>>>>>>
> > >>>>> some
> > >>>>>
> > >>>>>>> race
> > >>>>>>>
> > >>>>>>>> conditions because they may pass if I rerun them. Paulex, are
> > >>>>>>>>
> > >>> these
> > >>>
> > >>>>>>> tests
> > >>>>>>>
> > >>>>>>>> passing for you?
> > >>>>>>>>
> > >>>>>>>> And there are new 15 test failures. So now if I'll make a code
> > >>>>>>>>
> > >>>>> update
> > >>>>>
> > >>>>>>> or a
> > >>>>>>>
> > >>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
> > >>>>>>>>
> > >>>>>>>> Currently if a commit breaks the Harmony classlib build a
> > >>>>>>>>
> > >>>>> notification
> > >>>>>
> > >>>>>>> with
> > >>>>>>>
> > >>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
> > >>>>>>>>
> > >>> be
> > >>>
> > >>>>> send
> > >>>>>
> > >>>>>>> to
> > >>>>>>>
> > >>>>>>>> harmony-commits mailing list. Is it possible to have the same
> > >>>>>>>>
> > >>> for
> > >>>
> > >>>>> tests?
> > >>>>>
> > >>>>>>> I
> > >>>>>>>
> > >>>>>>>> mean that after completing automatic build the existing classlib
> > >>>>>>>>
> > >>>>> tests
> > >>>>>
> > >>>>>>> suite
> > >>>>>>>
> > >>>>>>>> should be run. If there are failing tests then a report
> > >>>>>>>>
> > >>> notification
> > >>>
> > >>>>>>> with
> > >>>>>>>
> > >>>>>>>> corresponding subject will be send.
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> Stepan Mishura
> > >>>>>>>> Intel Middleware Products Division
> > >>>>>>>>
> > >>>>>>> --
> > >>>>>>> Mark Hindess <ma...@googlemail.com>
> > >>>>>>> IBM Java Technology Centre, UK.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>> ---------------------------------------------------------------------
> > >>>
> > >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>>>>> To unsubscribe, e-mail:
> > >>>>>>>
> > >>> harmony-dev-unsubscribe@incubator.apache.org
> > >>>
> > >>>>>>> For additional commands, e-mail:
> > >>>>>>>
> > >>> harmony-dev-help@incubator.apache.org
> > >>>
> > >>>>>>>
> > >>>>>> --
> > >>>>>>
> > >>>>>>
> > >>> ---------------------------------------------------------------------
> > >>>
> > >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >>>>>> For additional commands, e-mail:
> > >>>>>>
> > >>> harmony-dev-help@incubator.apache.org
> > >>>
> > >>>>>> Thanks,
> > >>>>>> Stepan Mishura
> > >>>>>> Intel Middleware Products Division
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>> --
> > >>>>> Mark Hindess <ma...@googlemail.com>
> > >>>>> IBM Java Technology Centre, UK.
> > >>>>>
> > >>>>> ---------------------------------------------------------------------
> > >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>> --
> > >>>> ---------------------------------------------------------------------
> > >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >>>>
> > >>>> Thanks,
> > >>>> Stepan Mishura
> > >>>> Intel Middleware Products Division
> > >>>>
> > >>>>
> > >>>>
> > >>> --
> > >>> Mark Hindess <ma...@googlemail.com>
> > >>> IBM Java Technology Centre, UK.
> > >>>
> > >>> ---------------------------------------------------------------------
> > >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >>>
> > >>>
> > >>>
> > >>
> > >>
> > >> --
> > >> ---------------------------------------------------------------------
> > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >>
> > >> Thanks,
> > >> Stepan Mishura
> > >> Intel Middleware Products Division
> > >>
> > >>
> > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> --
> Mark Hindess <ma...@googlemail.com>
> IBM Java Technology Centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
Personally, obviously, I'd expect people to run the tests before committing.
However, I notice that since enabling the security tests - which fork
for every test - that the tests take over half an hour to run now on
our Linux build machine. So I can see why enthusiasm might lead to
people not running all the tests but instead perhaps just running
those from the module you are changing.
George just commented in a commit message backing out some changes in
text that all the text tests passed even though the changed caused
failures elsewhere.
Regards,
Mark.
On 4/11/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
> Just curious (and this isn't a criticism - I'm just as guilty of not
> doing this)...
>
> Don't you run the tests before committing?
>
> geir
>
> George Harley wrote:
> > Hi,
> >
> > It *seems* like things started failing after I committed the changes for
> > HARMONY-205 last night. I'm looking into this now. If the investigation
> > begins to take up too much time I will back the changes out.
> >
> > Best regards,
> > George
> >
> >
> > Stepan Mishura wrote:
> >> The same for me + DatagramChannelTest
> >>
> >> Thanks,
> >> Stepan.
> >>
> >> On 4/11/06, Mark Hindess wrote:
> >>
> >>> No. These:
> >>>
> >>> F
> >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
> >>>
> >>> E
> >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
> >>>
> >>> E
> >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
> >>>
> >>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
> >>> E
> >>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
> >>>
> >>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
> >>> F java.security.cert.X509CertSelectorTest.testMatch
> >>> F java.security.cert.X509CertSelectorTest.testClone
> >>> F tests.api.java.sql.DateTest.testSetTimelong
> >>> F tests.api.java.sql.DateTest.testToString
> >>> F tests.api.java.sql.DateTest.testValueOf
> >>> F tests.api.java.sql.TimestampTest.testSetNanosint
> >>> F tests.api.java.sql.TimestampTest.testToString
> >>> F tests.api.java.util.DateTest.test_toGMTString
> >>> F tests.api.java.util.DateTest.test_toString
> >>>
> >>> Regards,
> >>> Mark.
> >>>
> >>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> >>>
> >>>> On 4/11/06, Mark Hindess wrote:
> >>>>
> >>>>> Yes. I was using the failureproperty mechanism. Trying to get this
> >>>>> property propogated back to the top level ant file was what I was
> >>>>> having trouble with.
> >>>>>
> >>>>> Using a file as you suggest might help. I'll give that a try
> >>>>>
> >>> shortly...
> >>>
> >>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
> >>>>>
> >>>> I guess that you have 9 tests from DatagramChannelTest passed. And
> >>>> 12 +
> >>>>
> >>> 3 =
> >>>
> >>>> 15 :-)
> >>>>
> >>>> (And
> >>>>
> >>>>> there are typos - "mathc" should be "match" - in the failure messages
> >>>>> for java.security.cert testMatch and testClone.)
> >>>>>
> >>>> I've fixed typo.
> >>>>
> >>>> Thanks,
> >>>> Stepan.
> >>>>
> >>>> Regards,
> >>>>
> >>>>> Mark.
> >>>>>
> >>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> >>>>>
> >>>>>> On 4/11/06, Mark Hindess wrote:
> >>>>>>
> >>>>>>
> >>>>>>> Stepan,
> >>>>>>>
> >>>>>>> I have another build running (but without notifications going to
> >>>>>>>
> >>> the
> >>>
> >>>>>>> list) that runs:
> >>>>>>>
> >>>>>>> 1) build (with reference jdk)
> >>>>>>> 2) build with what we created with 1)
> >>>>>>> 3) test
> >>>>>>> 4) create classlists and compare with class load data for
> >>>>>>>
> >>> applications
> >>>
> >>>>>>> (tomcat, geronimo, continuum, etc.)
> >>>>>>> 5) generate JAPI results
> >>>>>>>
> >>>>>>> I'd like to fail this build at step 3, but I can't see an easy way
> >>>>>>>
> >>> to
> >>>
> >>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail if
> >>>>>>>
> >>> any
> >>>
> >>>>>>> of the module test sub-targets had test failures. I could parse
> >>>>>>>
> >>> the
> >>>
> >>>>>>> output I suppose, but I'd rather get ant to propagate the junit
> >>>>>>>
> >>> tasks
> >>>
> >>>>>>> failure property back up to the top level. I've tried a couple of
> >>>>>>> things and none seem to work. Any suggestions welcome.
> >>>>>>>
> >>>>>> Mark, did you try failureproperty parameter for junit task?
> >>>>>> We may add it to ant sub-targets to raise a flag, for example,
> >>>>>>
> >>> create
> >>>
> >>>>> file "
> >>>>>
> >>>>>> TESTS.FAILED" in the root, when tests for some module fail. And in
> >>>>>>
> >>> the
> >>>
> >>>>> end
> >>>>>
> >>>>>> of tests suite run check whether this file exists on not.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Stepan.
> >>>>>>
> >>>>>>
> >>>>>> Regards,
> >>>>>>
> >>>>>>> Mark.
> >>>>>>>
> >>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> I've checked out at morning last updates, built the code base
> >>>>>>>>
> >>> and
> >>>
> >>>>> run
> >>>>>
> >>>>>>> the
> >>>>>>>
> >>>>>>>> tests …and there are 24 tests failures!
> >>>>>>>>
> >>>>>>>> There are 9 tests failures in
> >>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest –
> >>>>>>>>
> >>> I
> >>>
> >>>>> saw
> >>>>>
> >>>>>>> these
> >>>>>>>
> >>>>>>>> failures before from time to time. It seems that tests depend on
> >>>>>>>>
> >>>>> some
> >>>>>
> >>>>>>> race
> >>>>>>>
> >>>>>>>> conditions because they may pass if I rerun them. Paulex, are
> >>>>>>>>
> >>> these
> >>>
> >>>>>>> tests
> >>>>>>>
> >>>>>>>> passing for you?
> >>>>>>>>
> >>>>>>>> And there are new 15 test failures. So now if I'll make a code
> >>>>>>>>
> >>>>> update
> >>>>>
> >>>>>>> or a
> >>>>>>>
> >>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
> >>>>>>>>
> >>>>>>>> Currently if a commit breaks the Harmony classlib build a
> >>>>>>>>
> >>>>> notification
> >>>>>
> >>>>>>> with
> >>>>>>>
> >>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
> >>>>>>>>
> >>> be
> >>>
> >>>>> send
> >>>>>
> >>>>>>> to
> >>>>>>>
> >>>>>>>> harmony-commits mailing list. Is it possible to have the same
> >>>>>>>>
> >>> for
> >>>
> >>>>> tests?
> >>>>>
> >>>>>>> I
> >>>>>>>
> >>>>>>>> mean that after completing automatic build the existing classlib
> >>>>>>>>
> >>>>> tests
> >>>>>
> >>>>>>> suite
> >>>>>>>
> >>>>>>>> should be run. If there are failing tests then a report
> >>>>>>>>
> >>> notification
> >>>
> >>>>>>> with
> >>>>>>>
> >>>>>>>> corresponding subject will be send.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Stepan Mishura
> >>>>>>>> Intel Middleware Products Division
> >>>>>>>>
> >>>>>>> --
> >>>>>>> Mark Hindess <ma...@googlemail.com>
> >>>>>>> IBM Java Technology Centre, UK.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>> ---------------------------------------------------------------------
> >>>
> >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>> To unsubscribe, e-mail:
> >>>>>>>
> >>> harmony-dev-unsubscribe@incubator.apache.org
> >>>
> >>>>>>> For additional commands, e-mail:
> >>>>>>>
> >>> harmony-dev-help@incubator.apache.org
> >>>
> >>>>>>>
> >>>>>> --
> >>>>>>
> >>>>>>
> >>> ---------------------------------------------------------------------
> >>>
> >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>>> For additional commands, e-mail:
> >>>>>>
> >>> harmony-dev-help@incubator.apache.org
> >>>
> >>>>>> Thanks,
> >>>>>> Stepan Mishura
> >>>>>> Intel Middleware Products Division
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> --
> >>>>> Mark Hindess <ma...@googlemail.com>
> >>>>> IBM Java Technology Centre, UK.
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>>
> >>>>>
> >>>>>
> >>>> --
> >>>> ---------------------------------------------------------------------
> >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>
> >>>> Thanks,
> >>>> Stepan Mishura
> >>>> Intel Middleware Products Division
> >>>>
> >>>>
> >>>>
> >>> --
> >>> Mark Hindess <ma...@googlemail.com>
> >>> IBM Java Technology Centre, UK.
> >>>
> >>> ---------------------------------------------------------------------
> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >> Thanks,
> >> Stepan Mishura
> >> Intel Middleware Products Division
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by George Harley <ge...@googlemail.com>.
Geir Magnusson Jr wrote:
> I forgot the smiley.
:-)
>
> I don't think this problem is so odd. Do you really think that
> side-effects like this will be that rare?
Seriously, I don't really know. The case we have been discussing today
was the first concrete example that I have encountered and I find the
prospect of other occurrences sufficiently worrying that I'm all in
favour of adding in some more Ant test targets along the lines of "test
this module's dependencies". As pointed out by others, it's pretty
daunting to have to run every test prior to a check-in and that is only
bound to worsen as we grow the code base.
That's why I think that Stepan's idea of just running the dependencies'
tests is a good one. It capitalizes on the componentized architecture of
the class libraries.
Best regards,
George
>
> geir
>
> Geir Magnusson Jr wrote:
>> So the answer is "no"?
>>
>> George Harley wrote:
>>> Geir Magnusson Jr wrote:
>>>> Just curious (and this isn't a criticism - I'm just as guilty of
>>>> not doing this)...
>>>>
>>>> Don't you run the tests before committing?
>>>
>>> Hi Geir,
>>>
>>> Depends what do you mean by "the tests".
>>>
>>> The change was completely encapsulated in the text component. I ran
>>> the text tests. I checked that the new tests introduced in the issue
>>> patch gave the same results when run against a 5.0 RI. All was fine.
>>>
>>> The automated build and test server (which runs all the tests in all
>>> the modules) found the problem. That is it's job and it did it well.
>>> The problem was found relatively quickly and has now been remedied.
>>> The turnaround time would have been quicker if the automated build
>>> and test server issued a failure notice on failing tests but that is
>>> now in the process of getting sorted out.
>>>
>>> IMHO this problem was a bit of an odd problem as the failures did
>>> not arise in the patched module's test suite but in other modules.
>>> That is a good indication that the test suite for text is in need of
>>> some scrutiny. That is a useful thing to be made aware of.
>>>
>>>
>>> Best regards,
>>> George
>>>
>>>>
>>>> geir
>>>>
>>>> George Harley wrote:
>>>>> Hi,
>>>>>
>>>>> It *seems* like things started failing after I committed the
>>>>> changes for HARMONY-205 last night. I'm looking into this now. If
>>>>> the investigation begins to take up too much time I will back the
>>>>> changes out.
>>>>>
>>>>> Best regards,
>>>>> George
>>>>>
>>>>>
>>>>> Stepan Mishura wrote:
>>>>>> The same for me + DatagramChannelTest
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan.
>>>>>>
>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>
>>>>>>> No. These:
>>>>>>>
>>>>>>> F
>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>>>>>>
>>>>>>> E
>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>>>>>>
>>>>>>> E
>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>>>>>>
>>>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>>>>> E
>>>>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>>>>>>
>>>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>>>>> F tests.api.java.sql.DateTest.testToString
>>>>>>> F tests.api.java.sql.DateTest.testValueOf
>>>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>>>>> F tests.api.java.sql.TimestampTest.testToString
>>>>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>>>>> F tests.api.java.util.DateTest.test_toString
>>>>>>>
>>>>>>> Regards,
>>>>>>> Mark.
>>>>>>>
>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get
>>>>>>>>> this
>>>>>>>>> property propogated back to the top level ant file was what I was
>>>>>>>>> having trouble with.
>>>>>>>>>
>>>>>>>>> Using a file as you suggest might help. I'll give that a try
>>>>>>> shortly...
>>>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>>>> I guess that you have 9 tests from DatagramChannelTest passed.
>>>>>>>> And 12 +
>>>>>>> 3 =
>>>>>>>> 15 :-)
>>>>>>>>
>>>>>>>> (And
>>>>>>>>> there are typos - "mathc" should be "match" - in the failure
>>>>>>>>> messages
>>>>>>>>> for java.security.cert testMatch and testClone.)
>>>>>>>> I've fixed typo.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>> Mark.
>>>>>>>>>
>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>>>
>>>>>>>>>>> Stepan,
>>>>>>>>>>>
>>>>>>>>>>> I have another build running (but without notifications
>>>>>>>>>>> going to
>>>>>>> the
>>>>>>>>>>> list) that runs:
>>>>>>>>>>>
>>>>>>>>>>> 1) build (with reference jdk)
>>>>>>>>>>> 2) build with what we created with 1)
>>>>>>>>>>> 3) test
>>>>>>>>>>> 4) create classlists and compare with class load data for
>>>>>>> applications
>>>>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>>>>> 5) generate JAPI results
>>>>>>>>>>>
>>>>>>>>>>> I'd like to fail this build at step 3, but I can't see an
>>>>>>>>>>> easy way
>>>>>>> to
>>>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then
>>>>>>>>>>> fail if
>>>>>>> any
>>>>>>>>>>> of the module test sub-targets had test failures. I could parse
>>>>>>> the
>>>>>>>>>>> output I suppose, but I'd rather get ant to propagate the junit
>>>>>>> tasks
>>>>>>>>>>> failure property back up to the top level. I've tried a
>>>>>>>>>>> couple of
>>>>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>>>> create
>>>>>>>>> file "
>>>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail.
>>>>>>>>>> And in
>>>>>>> the
>>>>>>>>> end
>>>>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Stepan.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>> Mark.
>>>>>>>>>>>
>>>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I've checked out at morning last updates, built the code base
>>>>>>> and
>>>>>>>>> run
>>>>>>>>>>> the
>>>>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>>>>
>>>>>>>>>>>> There are 9 tests failures in
>>>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
>>>>>>>>>>>> –
>>>>>>> I
>>>>>>>>> saw
>>>>>>>>>>> these
>>>>>>>>>>>> failures before from time to time. It seems that tests
>>>>>>>>>>>> depend on
>>>>>>>>> some
>>>>>>>>>>> race
>>>>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>>>> these
>>>>>>>>>>> tests
>>>>>>>>>>>> passing for you?
>>>>>>>>>>>>
>>>>>>>>>>>> And there are new 15 test failures. So now if I'll make a code
>>>>>>>>> update
>>>>>>>>>>> or a
>>>>>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>>>>>>
>>>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>>>> notification
>>>>>>>>>>> with
>>>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
>>>>>>> be
>>>>>>>>> send
>>>>>>>>>>> to
>>>>>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>>>> for
>>>>>>>>> tests?
>>>>>>>>>>> I
>>>>>>>>>>>> mean that after completing automatic build the existing
>>>>>>>>>>>> classlib
>>>>>>>>> tests
>>>>>>>>>>> suite
>>>>>>>>>>>> should be run. If there are failing tests then a report
>>>>>>> notification
>>>>>>>>>>> with
>>>>>>>>>>>> corresponding subject will be send.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Stepan Mishura
>>>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>> --
>>>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>> Thanks,
>>>>>>>>>> Stepan Mishura
>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail:
>>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan Mishura
>>>>>>>> Intel Middleware Products Division
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail:
>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan Mishura
>>>>>> Intel Middleware Products Division
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
I forgot the smiley.
I don't think this problem is so odd. Do you really think that
side-effects like this will be that rare?
geir
Geir Magnusson Jr wrote:
> So the answer is "no"?
>
> George Harley wrote:
>> Geir Magnusson Jr wrote:
>>> Just curious (and this isn't a criticism - I'm just as guilty of not
>>> doing this)...
>>>
>>> Don't you run the tests before committing?
>>
>> Hi Geir,
>>
>> Depends what do you mean by "the tests".
>>
>> The change was completely encapsulated in the text component. I ran
>> the text tests. I checked that the new tests introduced in the issue
>> patch gave the same results when run against a 5.0 RI. All was fine.
>>
>> The automated build and test server (which runs all the tests in all
>> the modules) found the problem. That is it's job and it did it well.
>> The problem was found relatively quickly and has now been remedied.
>> The turnaround time would have been quicker if the automated build and
>> test server issued a failure notice on failing tests but that is now
>> in the process of getting sorted out.
>>
>> IMHO this problem was a bit of an odd problem as the failures did not
>> arise in the patched module's test suite but in other modules. That is
>> a good indication that the test suite for text is in need of some
>> scrutiny. That is a useful thing to be made aware of.
>>
>>
>> Best regards,
>> George
>>
>>>
>>> geir
>>>
>>> George Harley wrote:
>>>> Hi,
>>>>
>>>> It *seems* like things started failing after I committed the changes
>>>> for HARMONY-205 last night. I'm looking into this now. If the
>>>> investigation begins to take up too much time I will back the
>>>> changes out.
>>>>
>>>> Best regards,
>>>> George
>>>>
>>>>
>>>> Stepan Mishura wrote:
>>>>> The same for me + DatagramChannelTest
>>>>>
>>>>> Thanks,
>>>>> Stepan.
>>>>>
>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>
>>>>>> No. These:
>>>>>>
>>>>>> F
>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>>>>>
>>>>>> E
>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>>>>>
>>>>>> E
>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>>>>>
>>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>>>> E
>>>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>>>>>
>>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>>>> F tests.api.java.sql.DateTest.testToString
>>>>>> F tests.api.java.sql.DateTest.testValueOf
>>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>>>> F tests.api.java.sql.TimestampTest.testToString
>>>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>>>> F tests.api.java.util.DateTest.test_toString
>>>>>>
>>>>>> Regards,
>>>>>> Mark.
>>>>>>
>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get this
>>>>>>>> property propogated back to the top level ant file was what I was
>>>>>>>> having trouble with.
>>>>>>>>
>>>>>>>> Using a file as you suggest might help. I'll give that a try
>>>>>> shortly...
>>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>>> I guess that you have 9 tests from DatagramChannelTest passed.
>>>>>>> And 12 +
>>>>>> 3 =
>>>>>>> 15 :-)
>>>>>>>
>>>>>>> (And
>>>>>>>> there are typos - "mathc" should be "match" - in the failure
>>>>>>>> messages
>>>>>>>> for java.security.cert testMatch and testClone.)
>>>>>>> I've fixed typo.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Stepan.
>>>>>>>
>>>>>>> Regards,
>>>>>>>> Mark.
>>>>>>>>
>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>>
>>>>>>>>>> Stepan,
>>>>>>>>>>
>>>>>>>>>> I have another build running (but without notifications going to
>>>>>> the
>>>>>>>>>> list) that runs:
>>>>>>>>>>
>>>>>>>>>> 1) build (with reference jdk)
>>>>>>>>>> 2) build with what we created with 1)
>>>>>>>>>> 3) test
>>>>>>>>>> 4) create classlists and compare with class load data for
>>>>>> applications
>>>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>>>> 5) generate JAPI results
>>>>>>>>>>
>>>>>>>>>> I'd like to fail this build at step 3, but I can't see an easy
>>>>>>>>>> way
>>>>>> to
>>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then
>>>>>>>>>> fail if
>>>>>> any
>>>>>>>>>> of the module test sub-targets had test failures. I could parse
>>>>>> the
>>>>>>>>>> output I suppose, but I'd rather get ant to propagate the junit
>>>>>> tasks
>>>>>>>>>> failure property back up to the top level. I've tried a couple of
>>>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>>> create
>>>>>>>> file "
>>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail. And in
>>>>>> the
>>>>>>>> end
>>>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Stepan.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>> Mark.
>>>>>>>>>>
>>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I've checked out at morning last updates, built the code base
>>>>>> and
>>>>>>>> run
>>>>>>>>>> the
>>>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>>>
>>>>>>>>>>> There are 9 tests failures in
>>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest –
>>>>>> I
>>>>>>>> saw
>>>>>>>>>> these
>>>>>>>>>>> failures before from time to time. It seems that tests depend on
>>>>>>>> some
>>>>>>>>>> race
>>>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>>> these
>>>>>>>>>> tests
>>>>>>>>>>> passing for you?
>>>>>>>>>>>
>>>>>>>>>>> And there are new 15 test failures. So now if I'll make a code
>>>>>>>> update
>>>>>>>>>> or a
>>>>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>>>>>
>>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>>> notification
>>>>>>>>>> with
>>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
>>>>>> be
>>>>>>>> send
>>>>>>>>>> to
>>>>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>>> for
>>>>>>>> tests?
>>>>>>>>>> I
>>>>>>>>>>> mean that after completing automatic build the existing classlib
>>>>>>>> tests
>>>>>>>>>> suite
>>>>>>>>>>> should be run. If there are failing tests then a report
>>>>>> notification
>>>>>>>>>> with
>>>>>>>>>>> corresponding subject will be send.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Stepan Mishura
>>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>> --
>>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>>
>>>>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>> To unsubscribe, e-mail:
>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>> --
>>>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>> Thanks,
>>>>>>>>> Stepan Mishura
>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Stepan Mishura
>>>>>>> Intel Middleware Products Division
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>> IBM Java Technology Centre, UK.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>
>>>>> Thanks,
>>>>> Stepan Mishura
>>>>> Intel Middleware Products Division
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by LvJimmy，Jing <fi...@gmail.com>.
In my opinion, the tests are independent from each other. So every test
start up with its own environment, and release the resources when finished.
In this way, there's no dependancy chain and no mess at all.
Yes, we've met serveral side effect testcases, how about making a list to
warn the developer? I have some of them:
��set SecurityManager and always set it back, and at this time it should
prove RuntimePermission("setSecurityManager")
��setup a server, bind it to a address, but does not close if test fails, so
add a "try...finally"
��so is other network service
��and file read/write
��more ...
2006/4/11, George Harley <ge...@googlemail.com>:
>
> Geir Magnusson Jr wrote:
> >
> >
> > George Harley wrote:
> >> Hi Geir,
> >>
> >> From my point of view the answer is yes, I ran the tests for the
> >> patched module. The build/test server picked up the downstream
> >> breakages in the other modules. I fixed matters so that the builds
> >> ran again.
> >
> > Right - I thought we considered the build/test server a safety to keep
> > accidents from getting away from us, but given the delay and the
> > distribution of people throughout the planet working on this (rather
> > than all sitting in the same office), we want to avoid breakages at
> > all costs. (Again, I'm guilty of the same thing...)
> >
> >>
> >> Elsewhere in this thread I am trying to cooperate with others to make
> >> it simpler to run more - just the necessary - tests prior to making a
> >> commit.
> >
> > I guess the question is "what is necessary"? Given a dependency chain
> >
> > A <- B <- C
> >
> > I don't think it's only the immediate dependencies of a given module
> > A, but rather the full dependency graph, because a "1 away" module (C)
> > might use a direct dependency (B) in a way different than the test
> > cases for that direct dependency do (especially since we aren't quite
> > up to speed with doing implementation testing...)
>
> Hi Geir,
>
> Sure, point taken. Testing A and then its set of immediate dependants
> (just B in this example) will greatly reduce the risk of downstream
> module breakages but will not completely catch all such events.
> Actually, subject to the quality of the tests in C, it is possible that
> even testing A, B and C will not find any such submerged problems.
>
> I am not an expert in the theory of testing ( as if you had to be told
> :-) ) but instinct alone tells me that making it convenient and quick to
> run a module's tests and those of its dependants will definitely
> *improve* matters - it would certainly have helped me last night before
> I broke the build.
>
> Best regards,
> George
>
> >
> >>
> >> Thank you for your support.
> >>
> >
> > Um...
> > geir
> >
> >>
> >> Best regards,
> >> George
> >>
> >>
> >> Geir Magnusson Jr wrote:
> >>> So the answer is "no"?
> >>>
> >>> George Harley wrote:
> >>>> Geir Magnusson Jr wrote:
> >>>>> Just curious (and this isn't a criticism - I'm just as guilty of
> >>>>> not doing this)...
> >>>>>
> >>>>> Don't you run the tests before committing?
> >>>>
> >>>> Hi Geir,
> >>>>
> >>>> Depends what do you mean by "the tests".
> >>>>
> >>>> The change was completely encapsulated in the text component. I ran
> >>>> the text tests. I checked that the new tests introduced in the
> >>>> issue patch gave the same results when run against a 5.0 RI. All
> >>>> was fine.
> >>>>
> >>>> The automated build and test server (which runs all the tests in
> >>>> all the modules) found the problem. That is it's job and it did it
> >>>> well. The problem was found relatively quickly and has now been
> >>>> remedied. The turnaround time would have been quicker if the
> >>>> automated build and test server issued a failure notice on failing
> >>>> tests but that is now in the process of getting sorted out.
> >>>>
> >>>> IMHO this problem was a bit of an odd problem as the failures did
> >>>> not arise in the patched module's test suite but in other modules.
> >>>> That is a good indication that the test suite for text is in need
> >>>> of some scrutiny. That is a useful thing to be made aware of.
> >>>>
> >>>>
> >>>> Best regards,
> >>>> George
> >>>>
> >>>>>
> >>>>> geir
> >>>>>
> >>>>> George Harley wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> It *seems* like things started failing after I committed the
> >>>>>> changes for HARMONY-205 last night. I'm looking into this now. If
> >>>>>> the investigation begins to take up too much time I will back the
> >>>>>> changes out.
> >>>>>>
> >>>>>> Best regards,
> >>>>>> George
> >>>>>>
> >>>>>>
> >>>>>> Stepan Mishura wrote:
> >>>>>>> The same for me + DatagramChannelTest
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Stepan.
> >>>>>>>
> >>>>>>> On 4/11/06, Mark Hindess wrote:
> >>>>>>>
> >>>>>>>> No. These:
> >>>>>>>>
> >>>>>>>> F
> >>>>>>>>
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
> >>>>>>>>
> >>>>>>>> E
> >>>>>>>>
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
> >>>>>>>>
> >>>>>>>> E
> >>>>>>>>
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
> >>>>>>>>
> >>>>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
> >>>>>>>> E
> >>>>>>>>
> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
> >>>>>>>>
> >>>>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
> >>>>>>>> F java.security.cert.X509CertSelectorTest.testMatch
> >>>>>>>> F java.security.cert.X509CertSelectorTest.testClone
> >>>>>>>> F tests.api.java.sql.DateTest.testSetTimelong
> >>>>>>>> F tests.api.java.sql.DateTest.testToString
> >>>>>>>> F tests.api.java.sql.DateTest.testValueOf
> >>>>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
> >>>>>>>> F tests.api.java.sql.TimestampTest.testToString
> >>>>>>>> F tests.api.java.util.DateTest.test_toGMTString
> >>>>>>>> F tests.api.java.util.DateTest.test_toString
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Mark.
> >>>>>>>>
> >>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> >>>>>>>>> On 4/11/06, Mark Hindess wrote:
> >>>>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get
> >>>>>>>>>> this
> >>>>>>>>>> property propogated back to the top level ant file was what I
> >>>>>>>>>> was
> >>>>>>>>>> having trouble with.
> >>>>>>>>>>
> >>>>>>>>>> Using a file as you suggest might help. I'll give that a try
> >>>>>>>> shortly...
> >>>>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
> >>>>>>>>> I guess that you have 9 tests from DatagramChannelTest passed.
> >>>>>>>>> And 12 +
> >>>>>>>> 3 =
> >>>>>>>>> 15 :-)
> >>>>>>>>>
> >>>>>>>>> (And
> >>>>>>>>>> there are typos - "mathc" should be "match" - in the failure
> >>>>>>>>>> messages
> >>>>>>>>>> for java.security.cert testMatch and testClone.)
> >>>>>>>>> I've fixed typo.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Stepan.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>>> Mark.
> >>>>>>>>>>
> >>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> >>>>>>>>>>> On 4/11/06, Mark Hindess wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Stepan,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I have another build running (but without notifications
> >>>>>>>>>>>> going to
> >>>>>>>> the
> >>>>>>>>>>>> list) that runs:
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1) build (with reference jdk)
> >>>>>>>>>>>> 2) build with what we created with 1)
> >>>>>>>>>>>> 3) test
> >>>>>>>>>>>> 4) create classlists and compare with class load data for
> >>>>>>>> applications
> >>>>>>>>>>>> (tomcat, geronimo, continuum, etc.)
> >>>>>>>>>>>> 5) generate JAPI results
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'd like to fail this build at step 3, but I can't see an
> >>>>>>>>>>>> easy way
> >>>>>>>> to
> >>>>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then
> >>>>>>>>>>>> fail if
> >>>>>>>> any
> >>>>>>>>>>>> of the module test sub-targets had test failures. I could
> >>>>>>>>>>>> parse
> >>>>>>>> the
> >>>>>>>>>>>> output I suppose, but I'd rather get ant to propagate the
> >>>>>>>>>>>> junit
> >>>>>>>> tasks
> >>>>>>>>>>>> failure property back up to the top level. I've tried a
> >>>>>>>>>>>> couple of
> >>>>>>>>>>>> things and none seem to work. Any suggestions welcome.
> >>>>>>>>>>> Mark, did you try failureproperty parameter for junit task?
> >>>>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
> >>>>>>>> create
> >>>>>>>>>> file "
> >>>>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail.
> >>>>>>>>>>> And in
> >>>>>>>> the
> >>>>>>>>>> end
> >>>>>>>>>>> of tests suite run check whether this file exists on not.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Stepan.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Regards,
> >>>>>>>>>>>> Mark.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> >>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I've checked out at morning last updates, built the code
> base
> >>>>>>>> and
> >>>>>>>>>> run
> >>>>>>>>>>>> the
> >>>>>>>>>>>>> tests ��and there are 24 tests failures!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> There are 9 tests failures in
> >>>>>>>>>>>>>
> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
> >>>>>>>>>>>>> �C
> >>>>>>>> I
> >>>>>>>>>> saw
> >>>>>>>>>>>> these
> >>>>>>>>>>>>> failures before from time to time. It seems that tests
> >>>>>>>>>>>>> depend on
> >>>>>>>>>> some
> >>>>>>>>>>>> race
> >>>>>>>>>>>>> conditions because they may pass if I rerun them. Paulex,
> are
> >>>>>>>> these
> >>>>>>>>>>>> tests
> >>>>>>>>>>>>> passing for you?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> And there are new 15 test failures. So now if I'll make a
> >>>>>>>>>>>>> code
> >>>>>>>>>> update
> >>>>>>>>>>>> or a
> >>>>>>>>>>>>> bug fix how I can be 100% sure that I don't do any
> >>>>>>>>>>>>> regression?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
> >>>>>>>>>> notification
> >>>>>>>>>>>> with
> >>>>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32"
> >>>>>>>>>>>>> will
> >>>>>>>> be
> >>>>>>>>>> send
> >>>>>>>>>>>> to
> >>>>>>>>>>>>> harmony-commits mailing list. Is it possible to have the
> same
> >>>>>>>> for
> >>>>>>>>>> tests?
> >>>>>>>>>>>> I
> >>>>>>>>>>>>> mean that after completing automatic build the existing
> >>>>>>>>>>>>> classlib
> >>>>>>>>>> tests
> >>>>>>>>>>>> suite
> >>>>>>>>>>>>> should be run. If there are failing tests then a report
> >>>>>>>> notification
> >>>>>>>>>>>> with
> >>>>>>>>>>>>> corresponding subject will be send.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Stepan Mishura
> >>>>>>>>>>>>> Intel Middleware Products Division
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Mark Hindess <ma...@googlemail.com>
> >>>>>>>>>>>> IBM Java Technology Centre, UK.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>
> ---------------------------------------------------------------------
> >>>>>>>>
> >>>>>>>>>>>> Terms of use :
> >>>>>>>>>>>> http://incubator.apache.org/harmony/mailing.html
> >>>>>>>>>>>> To unsubscribe, e-mail:
> >>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>>>>>>> For additional commands, e-mail:
> >>>>>>>> harmony-dev-help@incubator.apache.org
> >>>>>>>>>>> --
> >>>>>>>>>>>
> >>>>>>>>
> ---------------------------------------------------------------------
> >>>>>>>>
> >>>>>>>>>>> Terms of use :
> http://incubator.apache.org/harmony/mailing.html
> >>>>>>>>>>> To unsubscribe, e-mail:
> >>>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>>>>>> For additional commands, e-mail:
> >>>>>>>> harmony-dev-help@incubator.apache.org
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Stepan Mishura
> >>>>>>>>>>> Intel Middleware Products Division
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Mark Hindess <ma...@googlemail.com>
> >>>>>>>>>> IBM Java Technology Centre, UK.
> >>>>>>>>>>
> >>>>>>>>>>
> ---------------------------------------------------------------------
> >>>>>>>>>>
> >>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>>>>> To unsubscribe, e-mail:
> >>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>>>>> For additional commands, e-mail:
> >>>>>>>>>> harmony-dev-help@incubator.apache.org
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>>
> ---------------------------------------------------------------------
> >>>>>>>>>
> >>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>>>> To unsubscribe, e-mail:
> >>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>>>> For additional commands, e-mail:
> >>>>>>>>> harmony-dev-help@incubator.apache.org
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Stepan Mishura
> >>>>>>>>> Intel Middleware Products Division
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> --
> >>>>>>>> Mark Hindess <ma...@googlemail.com>
> >>>>>>>> IBM Java Technology Centre, UK.
> >>>>>>>>
> >>>>>>>>
> ---------------------------------------------------------------------
> >>>>>>>>
> >>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>>> To unsubscribe, e-mail:
> >>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>>> For additional commands, e-mail:
> >>>>>>>> harmony-dev-help@incubator.apache.org
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>>
> ---------------------------------------------------------------------
> >>>>>>>
> >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>> To unsubscribe, e-mail:
> >>>>>>> harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>> For additional commands, e-mail:
> >>>>>>> harmony-dev-help@incubator.apache.org
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Stepan Mishura
> >>>>>>> Intel Middleware Products Division
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> ---------------------------------------------------------------------
> >>>>>>
> >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> >>>>>> For additional commands, e-mail:
> >>>>>> harmony-dev-help@incubator.apache.org
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> ---------------------------------------------------------------------
> >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>> For additional commands, e-mail:
> >>>>> harmony-dev-help@incubator.apache.org
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Best Regards!
Jimmy, Jing Lv
China Software Development Lab, IBM
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
Mark Hindess wrote:
> On 4/11/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
>> There is a cultural element to it - maybe we keep tabs on who breaks the
>> build, and that decides who buys the beer next time we all meet...
>
> Not that I object to George buying me beer, but this seems a little
> harsh considering he only broke 15 tests out of ~5400. Would he have
> to buy everyone dinner if he broken them all? ;-)
>
My theory is that over time, it will balance out - we'll all break
things, and we'll all wind up buying each other beer (as it should be)
geir
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
On 4/11/06, Geir Magnusson Jr <ge...@pobox.com> wrote:
>
> George Harley wrote:
> > Geir Magnusson Jr wrote:
> >>
> >>
> >> George Harley wrote:
> >>> Hi Geir,
> >>>
> >>> From my point of view the answer is yes, I ran the tests for the
> >>> patched module. The build/test server picked up the downstream
> >>> breakages in the other modules. I fixed matters so that the builds
> >>> ran again.
> >>
> >> Right - I thought we considered the build/test server a safety to keep
> >> accidents from getting away from us, but given the delay and the
> >> distribution of people throughout the planet working on this (rather
> >> than all sitting in the same office), we want to avoid breakages at
> >> all costs. (Again, I'm guilty of the same thing...)
> >>
> >>>
> >>> Elsewhere in this thread I am trying to cooperate with others to make
> >>> it simpler to run more - just the necessary - tests prior to making a
> >>> commit.
> >>
> >> I guess the question is "what is necessary"? Given a dependency chain
> >>
> >> A <- B <- C
> >>
> >> I don't think it's only the immediate dependencies of a given module
> >> A, but rather the full dependency graph, because a "1 away" module (C)
> >> might use a direct dependency (B) in a way different than the test
> >> cases for that direct dependency do (especially since we aren't quite
> >> up to speed with doing implementation testing...)
> >
> > Hi Geir,
> >
> > Sure, point taken. Testing A and then its set of immediate dependants
> > (just B in this example) will greatly reduce the risk of downstream
> > module breakages but will not completely catch all such events.
> > Actually, subject to the quality of the tests in C, it is possible that
> > even testing A, B and C will not find any such submerged problems.
>
> Indeed.
>
> >
> > I am not an expert in the theory of testing ( as if you had to be told
> > :-) ) but instinct alone tells me that making it convenient and quick to
> > run a module's tests and those of its dependants will definitely
> > *improve* matters - it would certainly have helped me last night before
> > I broke the build.
>
> There is a cultural element to it - maybe we keep tabs on who breaks the
> build, and that decides who buys the beer next time we all meet...
Not that I object to George buying me beer, but this seems a little
harsh considering he only broke 15 tests out of ~5400. Would he have
to buy everyone dinner if he broken them all? ;-)
Regards,
-Mark.
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
George Harley wrote:
> Geir Magnusson Jr wrote:
>>
>>
>> George Harley wrote:
>>> Hi Geir,
>>>
>>> From my point of view the answer is yes, I ran the tests for the
>>> patched module. The build/test server picked up the downstream
>>> breakages in the other modules. I fixed matters so that the builds
>>> ran again.
>>
>> Right - I thought we considered the build/test server a safety to keep
>> accidents from getting away from us, but given the delay and the
>> distribution of people throughout the planet working on this (rather
>> than all sitting in the same office), we want to avoid breakages at
>> all costs. (Again, I'm guilty of the same thing...)
>>
>>>
>>> Elsewhere in this thread I am trying to cooperate with others to make
>>> it simpler to run more - just the necessary - tests prior to making a
>>> commit.
>>
>> I guess the question is "what is necessary"? Given a dependency chain
>>
>> A <- B <- C
>>
>> I don't think it's only the immediate dependencies of a given module
>> A, but rather the full dependency graph, because a "1 away" module (C)
>> might use a direct dependency (B) in a way different than the test
>> cases for that direct dependency do (especially since we aren't quite
>> up to speed with doing implementation testing...)
>
> Hi Geir,
>
> Sure, point taken. Testing A and then its set of immediate dependants
> (just B in this example) will greatly reduce the risk of downstream
> module breakages but will not completely catch all such events.
> Actually, subject to the quality of the tests in C, it is possible that
> even testing A, B and C will not find any such submerged problems.
Indeed.
>
> I am not an expert in the theory of testing ( as if you had to be told
> :-) ) but instinct alone tells me that making it convenient and quick to
> run a module's tests and those of its dependants will definitely
> *improve* matters - it would certainly have helped me last night before
> I broke the build.
There is a cultural element to it - maybe we keep tabs on who breaks the
build, and that decides who buys the beer next time we all meet...
geir
>
> Best regards,
> George
>
>>
>>>
>>> Thank you for your support.
>>>
>>
>> Um...
>> geir
>>
>>>
>>> Best regards,
>>> George
>>>
>>>
>>> Geir Magnusson Jr wrote:
>>>> So the answer is "no"?
>>>>
>>>> George Harley wrote:
>>>>> Geir Magnusson Jr wrote:
>>>>>> Just curious (and this isn't a criticism - I'm just as guilty of
>>>>>> not doing this)...
>>>>>>
>>>>>> Don't you run the tests before committing?
>>>>>
>>>>> Hi Geir,
>>>>>
>>>>> Depends what do you mean by "the tests".
>>>>>
>>>>> The change was completely encapsulated in the text component. I ran
>>>>> the text tests. I checked that the new tests introduced in the
>>>>> issue patch gave the same results when run against a 5.0 RI. All
>>>>> was fine.
>>>>>
>>>>> The automated build and test server (which runs all the tests in
>>>>> all the modules) found the problem. That is it's job and it did it
>>>>> well. The problem was found relatively quickly and has now been
>>>>> remedied. The turnaround time would have been quicker if the
>>>>> automated build and test server issued a failure notice on failing
>>>>> tests but that is now in the process of getting sorted out.
>>>>>
>>>>> IMHO this problem was a bit of an odd problem as the failures did
>>>>> not arise in the patched module's test suite but in other modules.
>>>>> That is a good indication that the test suite for text is in need
>>>>> of some scrutiny. That is a useful thing to be made aware of.
>>>>>
>>>>>
>>>>> Best regards,
>>>>> George
>>>>>
>>>>>>
>>>>>> geir
>>>>>>
>>>>>> George Harley wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> It *seems* like things started failing after I committed the
>>>>>>> changes for HARMONY-205 last night. I'm looking into this now. If
>>>>>>> the investigation begins to take up too much time I will back the
>>>>>>> changes out.
>>>>>>>
>>>>>>> Best regards,
>>>>>>> George
>>>>>>>
>>>>>>>
>>>>>>> Stepan Mishura wrote:
>>>>>>>> The same for me + DatagramChannelTest
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan.
>>>>>>>>
>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>
>>>>>>>>> No. These:
>>>>>>>>>
>>>>>>>>> F
>>>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>>>>>>>>
>>>>>>>>> E
>>>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>>>>>>>>
>>>>>>>>> E
>>>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>>>>>>>>
>>>>>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>>>>>>> E
>>>>>>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>>>>>>>>
>>>>>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>>>>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>>>>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>>>>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>>>>>>> F tests.api.java.sql.DateTest.testToString
>>>>>>>>> F tests.api.java.sql.DateTest.testValueOf
>>>>>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>>>>>>> F tests.api.java.sql.TimestampTest.testToString
>>>>>>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>>>>>>> F tests.api.java.util.DateTest.test_toString
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Mark.
>>>>>>>>>
>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get
>>>>>>>>>>> this
>>>>>>>>>>> property propogated back to the top level ant file was what I
>>>>>>>>>>> was
>>>>>>>>>>> having trouble with.
>>>>>>>>>>>
>>>>>>>>>>> Using a file as you suggest might help. I'll give that a try
>>>>>>>>> shortly...
>>>>>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>>>>>> I guess that you have 9 tests from DatagramChannelTest passed.
>>>>>>>>>> And 12 +
>>>>>>>>> 3 =
>>>>>>>>>> 15 :-)
>>>>>>>>>>
>>>>>>>>>> (And
>>>>>>>>>>> there are typos - "mathc" should be "match" - in the failure
>>>>>>>>>>> messages
>>>>>>>>>>> for java.security.cert testMatch and testClone.)
>>>>>>>>>> I've fixed typo.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Stepan.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>> Mark.
>>>>>>>>>>>
>>>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Stepan,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have another build running (but without notifications
>>>>>>>>>>>>> going to
>>>>>>>>> the
>>>>>>>>>>>>> list) that runs:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) build (with reference jdk)
>>>>>>>>>>>>> 2) build with what we created with 1)
>>>>>>>>>>>>> 3) test
>>>>>>>>>>>>> 4) create classlists and compare with class load data for
>>>>>>>>> applications
>>>>>>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>>>>>>> 5) generate JAPI results
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'd like to fail this build at step 3, but I can't see an
>>>>>>>>>>>>> easy way
>>>>>>>>> to
>>>>>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then
>>>>>>>>>>>>> fail if
>>>>>>>>> any
>>>>>>>>>>>>> of the module test sub-targets had test failures. I could
>>>>>>>>>>>>> parse
>>>>>>>>> the
>>>>>>>>>>>>> output I suppose, but I'd rather get ant to propagate the
>>>>>>>>>>>>> junit
>>>>>>>>> tasks
>>>>>>>>>>>>> failure property back up to the top level. I've tried a
>>>>>>>>>>>>> couple of
>>>>>>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>>>>>> create
>>>>>>>>>>> file "
>>>>>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail.
>>>>>>>>>>>> And in
>>>>>>>>> the
>>>>>>>>>>> end
>>>>>>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Stepan.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Mark.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I've checked out at morning last updates, built the code base
>>>>>>>>> and
>>>>>>>>>>> run
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There are 9 tests failures in
>>>>>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
>>>>>>>>>>>>>> –
>>>>>>>>> I
>>>>>>>>>>> saw
>>>>>>>>>>>>> these
>>>>>>>>>>>>>> failures before from time to time. It seems that tests
>>>>>>>>>>>>>> depend on
>>>>>>>>>>> some
>>>>>>>>>>>>> race
>>>>>>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>>>>>> these
>>>>>>>>>>>>> tests
>>>>>>>>>>>>>> passing for you?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And there are new 15 test failures. So now if I'll make a
>>>>>>>>>>>>>> code
>>>>>>>>>>> update
>>>>>>>>>>>>> or a
>>>>>>>>>>>>>> bug fix how I can be 100% sure that I don't do any
>>>>>>>>>>>>>> regression?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>>>>>> notification
>>>>>>>>>>>>> with
>>>>>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32"
>>>>>>>>>>>>>> will
>>>>>>>>> be
>>>>>>>>>>> send
>>>>>>>>>>>>> to
>>>>>>>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>>>>>> for
>>>>>>>>>>> tests?
>>>>>>>>>>>>> I
>>>>>>>>>>>>>> mean that after completing automatic build the existing
>>>>>>>>>>>>>> classlib
>>>>>>>>>>> tests
>>>>>>>>>>>>> suite
>>>>>>>>>>>>>> should be run. If there are failing tests then a report
>>>>>>>>> notification
>>>>>>>>>>>>> with
>>>>>>>>>>>>>> corresponding subject will be send.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Stepan Mishura
>>>>>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>>>>>> Terms of use :
>>>>>>>>>>>>> http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>>>> --
>>>>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Stepan Mishura
>>>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>>>
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>
>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Stepan Mishura
>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail:
>>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan Mishura
>>>>>>>> Intel Middleware Products Division
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by George Harley <ge...@googlemail.com>.
Geir Magnusson Jr wrote:
>
>
> George Harley wrote:
>> Hi Geir,
>>
>> From my point of view the answer is yes, I ran the tests for the
>> patched module. The build/test server picked up the downstream
>> breakages in the other modules. I fixed matters so that the builds
>> ran again.
>
> Right - I thought we considered the build/test server a safety to keep
> accidents from getting away from us, but given the delay and the
> distribution of people throughout the planet working on this (rather
> than all sitting in the same office), we want to avoid breakages at
> all costs. (Again, I'm guilty of the same thing...)
>
>>
>> Elsewhere in this thread I am trying to cooperate with others to make
>> it simpler to run more - just the necessary - tests prior to making a
>> commit.
>
> I guess the question is "what is necessary"? Given a dependency chain
>
> A <- B <- C
>
> I don't think it's only the immediate dependencies of a given module
> A, but rather the full dependency graph, because a "1 away" module (C)
> might use a direct dependency (B) in a way different than the test
> cases for that direct dependency do (especially since we aren't quite
> up to speed with doing implementation testing...)
Hi Geir,
Sure, point taken. Testing A and then its set of immediate dependants
(just B in this example) will greatly reduce the risk of downstream
module breakages but will not completely catch all such events.
Actually, subject to the quality of the tests in C, it is possible that
even testing A, B and C will not find any such submerged problems.
I am not an expert in the theory of testing ( as if you had to be told
:-) ) but instinct alone tells me that making it convenient and quick to
run a module's tests and those of its dependants will definitely
*improve* matters - it would certainly have helped me last night before
I broke the build.
Best regards,
George
>
>>
>> Thank you for your support.
>>
>
> Um...
> geir
>
>>
>> Best regards,
>> George
>>
>>
>> Geir Magnusson Jr wrote:
>>> So the answer is "no"?
>>>
>>> George Harley wrote:
>>>> Geir Magnusson Jr wrote:
>>>>> Just curious (and this isn't a criticism - I'm just as guilty of
>>>>> not doing this)...
>>>>>
>>>>> Don't you run the tests before committing?
>>>>
>>>> Hi Geir,
>>>>
>>>> Depends what do you mean by "the tests".
>>>>
>>>> The change was completely encapsulated in the text component. I ran
>>>> the text tests. I checked that the new tests introduced in the
>>>> issue patch gave the same results when run against a 5.0 RI. All
>>>> was fine.
>>>>
>>>> The automated build and test server (which runs all the tests in
>>>> all the modules) found the problem. That is it's job and it did it
>>>> well. The problem was found relatively quickly and has now been
>>>> remedied. The turnaround time would have been quicker if the
>>>> automated build and test server issued a failure notice on failing
>>>> tests but that is now in the process of getting sorted out.
>>>>
>>>> IMHO this problem was a bit of an odd problem as the failures did
>>>> not arise in the patched module's test suite but in other modules.
>>>> That is a good indication that the test suite for text is in need
>>>> of some scrutiny. That is a useful thing to be made aware of.
>>>>
>>>>
>>>> Best regards,
>>>> George
>>>>
>>>>>
>>>>> geir
>>>>>
>>>>> George Harley wrote:
>>>>>> Hi,
>>>>>>
>>>>>> It *seems* like things started failing after I committed the
>>>>>> changes for HARMONY-205 last night. I'm looking into this now. If
>>>>>> the investigation begins to take up too much time I will back the
>>>>>> changes out.
>>>>>>
>>>>>> Best regards,
>>>>>> George
>>>>>>
>>>>>>
>>>>>> Stepan Mishura wrote:
>>>>>>> The same for me + DatagramChannelTest
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Stepan.
>>>>>>>
>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>
>>>>>>>> No. These:
>>>>>>>>
>>>>>>>> F
>>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>>>>>>>
>>>>>>>> E
>>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>>>>>>>
>>>>>>>> E
>>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>>>>>>>
>>>>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>>>>>> E
>>>>>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>>>>>>>
>>>>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>>>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>>>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>>>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>>>>>> F tests.api.java.sql.DateTest.testToString
>>>>>>>> F tests.api.java.sql.DateTest.testValueOf
>>>>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>>>>>> F tests.api.java.sql.TimestampTest.testToString
>>>>>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>>>>>> F tests.api.java.util.DateTest.test_toString
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Mark.
>>>>>>>>
>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get
>>>>>>>>>> this
>>>>>>>>>> property propogated back to the top level ant file was what I
>>>>>>>>>> was
>>>>>>>>>> having trouble with.
>>>>>>>>>>
>>>>>>>>>> Using a file as you suggest might help. I'll give that a try
>>>>>>>> shortly...
>>>>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>>>>> I guess that you have 9 tests from DatagramChannelTest passed.
>>>>>>>>> And 12 +
>>>>>>>> 3 =
>>>>>>>>> 15 :-)
>>>>>>>>>
>>>>>>>>> (And
>>>>>>>>>> there are typos - "mathc" should be "match" - in the failure
>>>>>>>>>> messages
>>>>>>>>>> for java.security.cert testMatch and testClone.)
>>>>>>>>> I've fixed typo.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Stepan.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>> Mark.
>>>>>>>>>>
>>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Stepan,
>>>>>>>>>>>>
>>>>>>>>>>>> I have another build running (but without notifications
>>>>>>>>>>>> going to
>>>>>>>> the
>>>>>>>>>>>> list) that runs:
>>>>>>>>>>>>
>>>>>>>>>>>> 1) build (with reference jdk)
>>>>>>>>>>>> 2) build with what we created with 1)
>>>>>>>>>>>> 3) test
>>>>>>>>>>>> 4) create classlists and compare with class load data for
>>>>>>>> applications
>>>>>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>>>>>> 5) generate JAPI results
>>>>>>>>>>>>
>>>>>>>>>>>> I'd like to fail this build at step 3, but I can't see an
>>>>>>>>>>>> easy way
>>>>>>>> to
>>>>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then
>>>>>>>>>>>> fail if
>>>>>>>> any
>>>>>>>>>>>> of the module test sub-targets had test failures. I could
>>>>>>>>>>>> parse
>>>>>>>> the
>>>>>>>>>>>> output I suppose, but I'd rather get ant to propagate the
>>>>>>>>>>>> junit
>>>>>>>> tasks
>>>>>>>>>>>> failure property back up to the top level. I've tried a
>>>>>>>>>>>> couple of
>>>>>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>>>>> create
>>>>>>>>>> file "
>>>>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail.
>>>>>>>>>>> And in
>>>>>>>> the
>>>>>>>>>> end
>>>>>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Stepan.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>> Mark.
>>>>>>>>>>>>
>>>>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've checked out at morning last updates, built the code base
>>>>>>>> and
>>>>>>>>>> run
>>>>>>>>>>>> the
>>>>>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>>>>>
>>>>>>>>>>>>> There are 9 tests failures in
>>>>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
>>>>>>>>>>>>> –
>>>>>>>> I
>>>>>>>>>> saw
>>>>>>>>>>>> these
>>>>>>>>>>>>> failures before from time to time. It seems that tests
>>>>>>>>>>>>> depend on
>>>>>>>>>> some
>>>>>>>>>>>> race
>>>>>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>>>>> these
>>>>>>>>>>>> tests
>>>>>>>>>>>>> passing for you?
>>>>>>>>>>>>>
>>>>>>>>>>>>> And there are new 15 test failures. So now if I'll make a
>>>>>>>>>>>>> code
>>>>>>>>>> update
>>>>>>>>>>>> or a
>>>>>>>>>>>>> bug fix how I can be 100% sure that I don't do any
>>>>>>>>>>>>> regression?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>>>>> notification
>>>>>>>>>>>> with
>>>>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32"
>>>>>>>>>>>>> will
>>>>>>>> be
>>>>>>>>>> send
>>>>>>>>>>>> to
>>>>>>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>>>>> for
>>>>>>>>>> tests?
>>>>>>>>>>>> I
>>>>>>>>>>>>> mean that after completing automatic build the existing
>>>>>>>>>>>>> classlib
>>>>>>>>>> tests
>>>>>>>>>>>> suite
>>>>>>>>>>>>> should be run. If there are failing tests then a report
>>>>>>>> notification
>>>>>>>>>>>> with
>>>>>>>>>>>>> corresponding subject will be send.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Stepan Mishura
>>>>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>>> --
>>>>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>>>>>> Terms of use :
>>>>>>>>>>>> http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Stepan Mishura
>>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail:
>>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Stepan Mishura
>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail:
>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Stepan Mishura
>>>>>>> Intel Middleware Products Division
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
George Harley wrote:
> Hi Geir,
>
> From my point of view the answer is yes, I ran the tests for the
> patched module. The build/test server picked up the downstream breakages
> in the other modules. I fixed matters so that the builds ran again.
Right - I thought we considered the build/test server a safety to keep
accidents from getting away from us, but given the delay and the
distribution of people throughout the planet working on this (rather
than all sitting in the same office), we want to avoid breakages at all
costs. (Again, I'm guilty of the same thing...)
>
> Elsewhere in this thread I am trying to cooperate with others to make it
> simpler to run more - just the necessary - tests prior to making a commit.
I guess the question is "what is necessary"? Given a dependency chain
A <- B <- C
I don't think it's only the immediate dependencies of a given module A,
but rather the full dependency graph, because a "1 away" module (C)
might use a direct dependency (B) in a way different than the test cases
for that direct dependency do (especially since we aren't quite up to
speed with doing implementation testing...)
>
> Thank you for your support.
>
Um...
geir
>
> Best regards,
> George
>
>
> Geir Magnusson Jr wrote:
>> So the answer is "no"?
>>
>> George Harley wrote:
>>> Geir Magnusson Jr wrote:
>>>> Just curious (and this isn't a criticism - I'm just as guilty of not
>>>> doing this)...
>>>>
>>>> Don't you run the tests before committing?
>>>
>>> Hi Geir,
>>>
>>> Depends what do you mean by "the tests".
>>>
>>> The change was completely encapsulated in the text component. I ran
>>> the text tests. I checked that the new tests introduced in the issue
>>> patch gave the same results when run against a 5.0 RI. All was fine.
>>>
>>> The automated build and test server (which runs all the tests in all
>>> the modules) found the problem. That is it's job and it did it well.
>>> The problem was found relatively quickly and has now been remedied.
>>> The turnaround time would have been quicker if the automated build
>>> and test server issued a failure notice on failing tests but that is
>>> now in the process of getting sorted out.
>>>
>>> IMHO this problem was a bit of an odd problem as the failures did not
>>> arise in the patched module's test suite but in other modules. That
>>> is a good indication that the test suite for text is in need of some
>>> scrutiny. That is a useful thing to be made aware of.
>>>
>>>
>>> Best regards,
>>> George
>>>
>>>>
>>>> geir
>>>>
>>>> George Harley wrote:
>>>>> Hi,
>>>>>
>>>>> It *seems* like things started failing after I committed the
>>>>> changes for HARMONY-205 last night. I'm looking into this now. If
>>>>> the investigation begins to take up too much time I will back the
>>>>> changes out.
>>>>>
>>>>> Best regards,
>>>>> George
>>>>>
>>>>>
>>>>> Stepan Mishura wrote:
>>>>>> The same for me + DatagramChannelTest
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan.
>>>>>>
>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>
>>>>>>> No. These:
>>>>>>>
>>>>>>> F
>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>>>>>>
>>>>>>> E
>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>>>>>>
>>>>>>> E
>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>>>>>>
>>>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>>>>> E
>>>>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>>>>>>
>>>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>>>>> F tests.api.java.sql.DateTest.testToString
>>>>>>> F tests.api.java.sql.DateTest.testValueOf
>>>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>>>>> F tests.api.java.sql.TimestampTest.testToString
>>>>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>>>>> F tests.api.java.util.DateTest.test_toString
>>>>>>>
>>>>>>> Regards,
>>>>>>> Mark.
>>>>>>>
>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get this
>>>>>>>>> property propogated back to the top level ant file was what I was
>>>>>>>>> having trouble with.
>>>>>>>>>
>>>>>>>>> Using a file as you suggest might help. I'll give that a try
>>>>>>> shortly...
>>>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>>>> I guess that you have 9 tests from DatagramChannelTest passed.
>>>>>>>> And 12 +
>>>>>>> 3 =
>>>>>>>> 15 :-)
>>>>>>>>
>>>>>>>> (And
>>>>>>>>> there are typos - "mathc" should be "match" - in the failure
>>>>>>>>> messages
>>>>>>>>> for java.security.cert testMatch and testClone.)
>>>>>>>> I've fixed typo.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>> Mark.
>>>>>>>>>
>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>>>
>>>>>>>>>>> Stepan,
>>>>>>>>>>>
>>>>>>>>>>> I have another build running (but without notifications going to
>>>>>>> the
>>>>>>>>>>> list) that runs:
>>>>>>>>>>>
>>>>>>>>>>> 1) build (with reference jdk)
>>>>>>>>>>> 2) build with what we created with 1)
>>>>>>>>>>> 3) test
>>>>>>>>>>> 4) create classlists and compare with class load data for
>>>>>>> applications
>>>>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>>>>> 5) generate JAPI results
>>>>>>>>>>>
>>>>>>>>>>> I'd like to fail this build at step 3, but I can't see an
>>>>>>>>>>> easy way
>>>>>>> to
>>>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then
>>>>>>>>>>> fail if
>>>>>>> any
>>>>>>>>>>> of the module test sub-targets had test failures. I could parse
>>>>>>> the
>>>>>>>>>>> output I suppose, but I'd rather get ant to propagate the junit
>>>>>>> tasks
>>>>>>>>>>> failure property back up to the top level. I've tried a
>>>>>>>>>>> couple of
>>>>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>>>> create
>>>>>>>>> file "
>>>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail.
>>>>>>>>>> And in
>>>>>>> the
>>>>>>>>> end
>>>>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Stepan.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>> Mark.
>>>>>>>>>>>
>>>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I've checked out at morning last updates, built the code base
>>>>>>> and
>>>>>>>>> run
>>>>>>>>>>> the
>>>>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>>>>
>>>>>>>>>>>> There are 9 tests failures in
>>>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
>>>>>>>>>>>> –
>>>>>>> I
>>>>>>>>> saw
>>>>>>>>>>> these
>>>>>>>>>>>> failures before from time to time. It seems that tests
>>>>>>>>>>>> depend on
>>>>>>>>> some
>>>>>>>>>>> race
>>>>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>>>> these
>>>>>>>>>>> tests
>>>>>>>>>>>> passing for you?
>>>>>>>>>>>>
>>>>>>>>>>>> And there are new 15 test failures. So now if I'll make a code
>>>>>>>>> update
>>>>>>>>>>> or a
>>>>>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>>>>>>
>>>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>>>> notification
>>>>>>>>>>> with
>>>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
>>>>>>> be
>>>>>>>>> send
>>>>>>>>>>> to
>>>>>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>>>> for
>>>>>>>>> tests?
>>>>>>>>>>> I
>>>>>>>>>>>> mean that after completing automatic build the existing
>>>>>>>>>>>> classlib
>>>>>>>>> tests
>>>>>>>>>>> suite
>>>>>>>>>>>> should be run. If there are failing tests then a report
>>>>>>> notification
>>>>>>>>>>> with
>>>>>>>>>>>> corresponding subject will be send.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Stepan Mishura
>>>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>> --
>>>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>> Thanks,
>>>>>>>>>> Stepan Mishura
>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail:
>>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan Mishura
>>>>>>>> Intel Middleware Products Division
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> ---------------------------------------------------------------------
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan Mishura
>>>>>> Intel Middleware Products Division
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by George Harley <ge...@googlemail.com>.
Hi Geir,
From my point of view the answer is yes, I ran the tests for the
patched module. The build/test server picked up the downstream breakages
in the other modules. I fixed matters so that the builds ran again.
Elsewhere in this thread I am trying to cooperate with others to make it
simpler to run more - just the necessary - tests prior to making a commit.
Thank you for your support.
Best regards,
George
Geir Magnusson Jr wrote:
> So the answer is "no"?
>
> George Harley wrote:
>> Geir Magnusson Jr wrote:
>>> Just curious (and this isn't a criticism - I'm just as guilty of not
>>> doing this)...
>>>
>>> Don't you run the tests before committing?
>>
>> Hi Geir,
>>
>> Depends what do you mean by "the tests".
>>
>> The change was completely encapsulated in the text component. I ran
>> the text tests. I checked that the new tests introduced in the issue
>> patch gave the same results when run against a 5.0 RI. All was fine.
>>
>> The automated build and test server (which runs all the tests in all
>> the modules) found the problem. That is it's job and it did it well.
>> The problem was found relatively quickly and has now been remedied.
>> The turnaround time would have been quicker if the automated build
>> and test server issued a failure notice on failing tests but that is
>> now in the process of getting sorted out.
>>
>> IMHO this problem was a bit of an odd problem as the failures did not
>> arise in the patched module's test suite but in other modules. That
>> is a good indication that the test suite for text is in need of some
>> scrutiny. That is a useful thing to be made aware of.
>>
>>
>> Best regards,
>> George
>>
>>>
>>> geir
>>>
>>> George Harley wrote:
>>>> Hi,
>>>>
>>>> It *seems* like things started failing after I committed the
>>>> changes for HARMONY-205 last night. I'm looking into this now. If
>>>> the investigation begins to take up too much time I will back the
>>>> changes out.
>>>>
>>>> Best regards,
>>>> George
>>>>
>>>>
>>>> Stepan Mishura wrote:
>>>>> The same for me + DatagramChannelTest
>>>>>
>>>>> Thanks,
>>>>> Stepan.
>>>>>
>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>
>>>>>> No. These:
>>>>>>
>>>>>> F
>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>>>>>
>>>>>> E
>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>>>>>
>>>>>> E
>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>>>>>
>>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>>>> E
>>>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>>>>>
>>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>>>> F tests.api.java.sql.DateTest.testToString
>>>>>> F tests.api.java.sql.DateTest.testValueOf
>>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>>>> F tests.api.java.sql.TimestampTest.testToString
>>>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>>>> F tests.api.java.util.DateTest.test_toString
>>>>>>
>>>>>> Regards,
>>>>>> Mark.
>>>>>>
>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get this
>>>>>>>> property propogated back to the top level ant file was what I was
>>>>>>>> having trouble with.
>>>>>>>>
>>>>>>>> Using a file as you suggest might help. I'll give that a try
>>>>>> shortly...
>>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>>> I guess that you have 9 tests from DatagramChannelTest passed.
>>>>>>> And 12 +
>>>>>> 3 =
>>>>>>> 15 :-)
>>>>>>>
>>>>>>> (And
>>>>>>>> there are typos - "mathc" should be "match" - in the failure
>>>>>>>> messages
>>>>>>>> for java.security.cert testMatch and testClone.)
>>>>>>> I've fixed typo.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Stepan.
>>>>>>>
>>>>>>> Regards,
>>>>>>>> Mark.
>>>>>>>>
>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>>
>>>>>>>>>> Stepan,
>>>>>>>>>>
>>>>>>>>>> I have another build running (but without notifications going to
>>>>>> the
>>>>>>>>>> list) that runs:
>>>>>>>>>>
>>>>>>>>>> 1) build (with reference jdk)
>>>>>>>>>> 2) build with what we created with 1)
>>>>>>>>>> 3) test
>>>>>>>>>> 4) create classlists and compare with class load data for
>>>>>> applications
>>>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>>>> 5) generate JAPI results
>>>>>>>>>>
>>>>>>>>>> I'd like to fail this build at step 3, but I can't see an
>>>>>>>>>> easy way
>>>>>> to
>>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then
>>>>>>>>>> fail if
>>>>>> any
>>>>>>>>>> of the module test sub-targets had test failures. I could parse
>>>>>> the
>>>>>>>>>> output I suppose, but I'd rather get ant to propagate the junit
>>>>>> tasks
>>>>>>>>>> failure property back up to the top level. I've tried a
>>>>>>>>>> couple of
>>>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>>> create
>>>>>>>> file "
>>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail.
>>>>>>>>> And in
>>>>>> the
>>>>>>>> end
>>>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Stepan.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>> Mark.
>>>>>>>>>>
>>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I've checked out at morning last updates, built the code base
>>>>>> and
>>>>>>>> run
>>>>>>>>>> the
>>>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>>>
>>>>>>>>>>> There are 9 tests failures in
>>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
>>>>>>>>>>> –
>>>>>> I
>>>>>>>> saw
>>>>>>>>>> these
>>>>>>>>>>> failures before from time to time. It seems that tests
>>>>>>>>>>> depend on
>>>>>>>> some
>>>>>>>>>> race
>>>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>>> these
>>>>>>>>>> tests
>>>>>>>>>>> passing for you?
>>>>>>>>>>>
>>>>>>>>>>> And there are new 15 test failures. So now if I'll make a code
>>>>>>>> update
>>>>>>>>>> or a
>>>>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>>>>>
>>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>>> notification
>>>>>>>>>> with
>>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
>>>>>> be
>>>>>>>> send
>>>>>>>>>> to
>>>>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>>> for
>>>>>>>> tests?
>>>>>>>>>> I
>>>>>>>>>>> mean that after completing automatic build the existing
>>>>>>>>>>> classlib
>>>>>>>> tests
>>>>>>>>>> suite
>>>>>>>>>>> should be run. If there are failing tests then a report
>>>>>> notification
>>>>>>>>>> with
>>>>>>>>>>> corresponding subject will be send.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Stepan Mishura
>>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>> --
>>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>>
>>>>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>> To unsubscribe, e-mail:
>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>> --
>>>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>> Thanks,
>>>>>>>>> Stepan Mishura
>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail:
>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Stepan Mishura
>>>>>>> Intel Middleware Products Division
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>> IBM Java Technology Centre, UK.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>
>>>>> Thanks,
>>>>> Stepan Mishura
>>>>> Intel Middleware Products Division
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
So the answer is "no"?
George Harley wrote:
> Geir Magnusson Jr wrote:
>> Just curious (and this isn't a criticism - I'm just as guilty of not
>> doing this)...
>>
>> Don't you run the tests before committing?
>
> Hi Geir,
>
> Depends what do you mean by "the tests".
>
> The change was completely encapsulated in the text component. I ran the
> text tests. I checked that the new tests introduced in the issue patch
> gave the same results when run against a 5.0 RI. All was fine.
>
> The automated build and test server (which runs all the tests in all the
> modules) found the problem. That is it's job and it did it well. The
> problem was found relatively quickly and has now been remedied. The
> turnaround time would have been quicker if the automated build and test
> server issued a failure notice on failing tests but that is now in the
> process of getting sorted out.
>
> IMHO this problem was a bit of an odd problem as the failures did not
> arise in the patched module's test suite but in other modules. That is a
> good indication that the test suite for text is in need of some
> scrutiny. That is a useful thing to be made aware of.
>
>
> Best regards,
> George
>
>>
>> geir
>>
>> George Harley wrote:
>>> Hi,
>>>
>>> It *seems* like things started failing after I committed the changes
>>> for HARMONY-205 last night. I'm looking into this now. If the
>>> investigation begins to take up too much time I will back the changes
>>> out.
>>>
>>> Best regards,
>>> George
>>>
>>>
>>> Stepan Mishura wrote:
>>>> The same for me + DatagramChannelTest
>>>>
>>>> Thanks,
>>>> Stepan.
>>>>
>>>> On 4/11/06, Mark Hindess wrote:
>>>>
>>>>> No. These:
>>>>>
>>>>> F
>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>>>>
>>>>> E
>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>>>>
>>>>> E
>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>>>>
>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>>> E
>>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>>>>
>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>>> F tests.api.java.sql.DateTest.testToString
>>>>> F tests.api.java.sql.DateTest.testValueOf
>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>>> F tests.api.java.sql.TimestampTest.testToString
>>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>>> F tests.api.java.util.DateTest.test_toString
>>>>>
>>>>> Regards,
>>>>> Mark.
>>>>>
>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get this
>>>>>>> property propogated back to the top level ant file was what I was
>>>>>>> having trouble with.
>>>>>>>
>>>>>>> Using a file as you suggest might help. I'll give that a try
>>>>> shortly...
>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>> I guess that you have 9 tests from DatagramChannelTest passed. And
>>>>>> 12 +
>>>>> 3 =
>>>>>> 15 :-)
>>>>>>
>>>>>> (And
>>>>>>> there are typos - "mathc" should be "match" - in the failure
>>>>>>> messages
>>>>>>> for java.security.cert testMatch and testClone.)
>>>>>> I've fixed typo.
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan.
>>>>>>
>>>>>> Regards,
>>>>>>> Mark.
>>>>>>>
>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>
>>>>>>>>> Stepan,
>>>>>>>>>
>>>>>>>>> I have another build running (but without notifications going to
>>>>> the
>>>>>>>>> list) that runs:
>>>>>>>>>
>>>>>>>>> 1) build (with reference jdk)
>>>>>>>>> 2) build with what we created with 1)
>>>>>>>>> 3) test
>>>>>>>>> 4) create classlists and compare with class load data for
>>>>> applications
>>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>>> 5) generate JAPI results
>>>>>>>>>
>>>>>>>>> I'd like to fail this build at step 3, but I can't see an easy way
>>>>> to
>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail if
>>>>> any
>>>>>>>>> of the module test sub-targets had test failures. I could parse
>>>>> the
>>>>>>>>> output I suppose, but I'd rather get ant to propagate the junit
>>>>> tasks
>>>>>>>>> failure property back up to the top level. I've tried a couple of
>>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>> create
>>>>>>> file "
>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail. And in
>>>>> the
>>>>>>> end
>>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan.
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>> Mark.
>>>>>>>>>
>>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I've checked out at morning last updates, built the code base
>>>>> and
>>>>>>> run
>>>>>>>>> the
>>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>>
>>>>>>>>>> There are 9 tests failures in
>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest –
>>>>> I
>>>>>>> saw
>>>>>>>>> these
>>>>>>>>>> failures before from time to time. It seems that tests depend on
>>>>>>> some
>>>>>>>>> race
>>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>> these
>>>>>>>>> tests
>>>>>>>>>> passing for you?
>>>>>>>>>>
>>>>>>>>>> And there are new 15 test failures. So now if I'll make a code
>>>>>>> update
>>>>>>>>> or a
>>>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>>>>
>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>> notification
>>>>>>>>> with
>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
>>>>> be
>>>>>>> send
>>>>>>>>> to
>>>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>> for
>>>>>>> tests?
>>>>>>>>> I
>>>>>>>>>> mean that after completing automatic build the existing classlib
>>>>>>> tests
>>>>>>>>> suite
>>>>>>>>>> should be run. If there are failing tests then a report
>>>>> notification
>>>>>>>>> with
>>>>>>>>>> corresponding subject will be send.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Stepan Mishura
>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>> --
>>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>
>>>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail:
>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>> --
>>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>> Thanks,
>>>>>>>> Stepan Mishura
>>>>>>>> Intel Middleware Products Division
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> ---------------------------------------------------------------------
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan Mishura
>>>>>> Intel Middleware Products Division
>>>>>>
>>>>>>
>>>>> --
>>>>> Mark Hindess <ma...@googlemail.com>
>>>>> IBM Java Technology Centre, UK.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>> Thanks,
>>>> Stepan Mishura
>>>> Intel Middleware Products Division
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by George Harley <ge...@googlemail.com>.
Geir Magnusson Jr wrote:
> Just curious (and this isn't a criticism - I'm just as guilty of not
> doing this)...
>
> Don't you run the tests before committing?
Hi Geir,
Depends what do you mean by "the tests".
The change was completely encapsulated in the text component. I ran the
text tests. I checked that the new tests introduced in the issue patch
gave the same results when run against a 5.0 RI. All was fine.
The automated build and test server (which runs all the tests in all the
modules) found the problem. That is it's job and it did it well. The
problem was found relatively quickly and has now been remedied. The
turnaround time would have been quicker if the automated build and test
server issued a failure notice on failing tests but that is now in the
process of getting sorted out.
IMHO this problem was a bit of an odd problem as the failures did not
arise in the patched module's test suite but in other modules. That is a
good indication that the test suite for text is in need of some
scrutiny. That is a useful thing to be made aware of.
Best regards,
George
>
> geir
>
> George Harley wrote:
>> Hi,
>>
>> It *seems* like things started failing after I committed the changes
>> for HARMONY-205 last night. I'm looking into this now. If the
>> investigation begins to take up too much time I will back the changes
>> out.
>>
>> Best regards,
>> George
>>
>>
>> Stepan Mishura wrote:
>>> The same for me + DatagramChannelTest
>>>
>>> Thanks,
>>> Stepan.
>>>
>>> On 4/11/06, Mark Hindess wrote:
>>>
>>>> No. These:
>>>>
>>>> F
>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>>>
>>>> E
>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>>>
>>>> E
>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>>>
>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>> E
>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>>>
>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>> F tests.api.java.sql.DateTest.testToString
>>>> F tests.api.java.sql.DateTest.testValueOf
>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>> F tests.api.java.sql.TimestampTest.testToString
>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>> F tests.api.java.util.DateTest.test_toString
>>>>
>>>> Regards,
>>>> Mark.
>>>>
>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>> Yes. I was using the failureproperty mechanism. Trying to get this
>>>>>> property propogated back to the top level ant file was what I was
>>>>>> having trouble with.
>>>>>>
>>>>>> Using a file as you suggest might help. I'll give that a try
>>>> shortly...
>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>> I guess that you have 9 tests from DatagramChannelTest passed. And
>>>>> 12 +
>>>> 3 =
>>>>> 15 :-)
>>>>>
>>>>> (And
>>>>>> there are typos - "mathc" should be "match" - in the failure
>>>>>> messages
>>>>>> for java.security.cert testMatch and testClone.)
>>>>> I've fixed typo.
>>>>>
>>>>> Thanks,
>>>>> Stepan.
>>>>>
>>>>> Regards,
>>>>>> Mark.
>>>>>>
>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>
>>>>>>>> Stepan,
>>>>>>>>
>>>>>>>> I have another build running (but without notifications going to
>>>> the
>>>>>>>> list) that runs:
>>>>>>>>
>>>>>>>> 1) build (with reference jdk)
>>>>>>>> 2) build with what we created with 1)
>>>>>>>> 3) test
>>>>>>>> 4) create classlists and compare with class load data for
>>>> applications
>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>> 5) generate JAPI results
>>>>>>>>
>>>>>>>> I'd like to fail this build at step 3, but I can't see an easy way
>>>> to
>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail if
>>>> any
>>>>>>>> of the module test sub-targets had test failures. I could parse
>>>> the
>>>>>>>> output I suppose, but I'd rather get ant to propagate the junit
>>>> tasks
>>>>>>>> failure property back up to the top level. I've tried a couple of
>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>> create
>>>>>> file "
>>>>>>> TESTS.FAILED" in the root, when tests for some module fail. And in
>>>> the
>>>>>> end
>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Stepan.
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>>> Mark.
>>>>>>>>
>>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I've checked out at morning last updates, built the code base
>>>> and
>>>>>> run
>>>>>>>> the
>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>
>>>>>>>>> There are 9 tests failures in
>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest –
>>>> I
>>>>>> saw
>>>>>>>> these
>>>>>>>>> failures before from time to time. It seems that tests depend on
>>>>>> some
>>>>>>>> race
>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>> these
>>>>>>>> tests
>>>>>>>>> passing for you?
>>>>>>>>>
>>>>>>>>> And there are new 15 test failures. So now if I'll make a code
>>>>>> update
>>>>>>>> or a
>>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>>>
>>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>> notification
>>>>>>>> with
>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
>>>> be
>>>>>> send
>>>>>>>> to
>>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>> for
>>>>>> tests?
>>>>>>>> I
>>>>>>>>> mean that after completing automatic build the existing classlib
>>>>>> tests
>>>>>>>> suite
>>>>>>>>> should be run. If there are failing tests then a report
>>>> notification
>>>>>>>> with
>>>>>>>>> corresponding subject will be send.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Stepan Mishura
>>>>>>>>> Intel Middleware Products Division
>>>>>>>> --
>>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>
>>>>>>>>
>>>> ---------------------------------------------------------------------
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail:
>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>> harmony-dev-help@incubator.apache.org
>>>>>>> --
>>>>>>>
>>>> ---------------------------------------------------------------------
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail:
>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>>>> harmony-dev-help@incubator.apache.org
>>>>>>> Thanks,
>>>>>>> Stepan Mishura
>>>>>>> Intel Middleware Products Division
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>> IBM Java Technology Centre, UK.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>> --
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>
>>>>> Thanks,
>>>>> Stepan Mishura
>>>>> Intel Middleware Products Division
>>>>>
>>>>>
>>>> --
>>>> Mark Hindess <ma...@googlemail.com>
>>>> IBM Java Technology Centre, UK.
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>
>>>
>>> --
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>> Thanks,
>>> Stepan Mishura
>>> Intel Middleware Products Division
>>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Geir Magnusson Jr <ge...@pobox.com>.
Just curious (and this isn't a criticism - I'm just as guilty of not
doing this)...
Don't you run the tests before committing?
geir
George Harley wrote:
> Hi,
>
> It *seems* like things started failing after I committed the changes for
> HARMONY-205 last night. I'm looking into this now. If the investigation
> begins to take up too much time I will back the changes out.
>
> Best regards,
> George
>
>
> Stepan Mishura wrote:
>> The same for me + DatagramChannelTest
>>
>> Thanks,
>> Stepan.
>>
>> On 4/11/06, Mark Hindess wrote:
>>
>>> No. These:
>>>
>>> F
>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>>>
>>> E
>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>>>
>>> E
>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>>>
>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>> E
>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>>>
>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>> F java.security.cert.X509CertSelectorTest.testMatch
>>> F java.security.cert.X509CertSelectorTest.testClone
>>> F tests.api.java.sql.DateTest.testSetTimelong
>>> F tests.api.java.sql.DateTest.testToString
>>> F tests.api.java.sql.DateTest.testValueOf
>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>> F tests.api.java.sql.TimestampTest.testToString
>>> F tests.api.java.util.DateTest.test_toGMTString
>>> F tests.api.java.util.DateTest.test_toString
>>>
>>> Regards,
>>> Mark.
>>>
>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>
>>>> On 4/11/06, Mark Hindess wrote:
>>>>
>>>>> Yes. I was using the failureproperty mechanism. Trying to get this
>>>>> property propogated back to the top level ant file was what I was
>>>>> having trouble with.
>>>>>
>>>>> Using a file as you suggest might help. I'll give that a try
>>>>>
>>> shortly...
>>>
>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>
>>>> I guess that you have 9 tests from DatagramChannelTest passed. And
>>>> 12 +
>>>>
>>> 3 =
>>>
>>>> 15 :-)
>>>>
>>>> (And
>>>>
>>>>> there are typos - "mathc" should be "match" - in the failure messages
>>>>> for java.security.cert testMatch and testClone.)
>>>>>
>>>> I've fixed typo.
>>>>
>>>> Thanks,
>>>> Stepan.
>>>>
>>>> Regards,
>>>>
>>>>> Mark.
>>>>>
>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>
>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>
>>>>>>
>>>>>>> Stepan,
>>>>>>>
>>>>>>> I have another build running (but without notifications going to
>>>>>>>
>>> the
>>>
>>>>>>> list) that runs:
>>>>>>>
>>>>>>> 1) build (with reference jdk)
>>>>>>> 2) build with what we created with 1)
>>>>>>> 3) test
>>>>>>> 4) create classlists and compare with class load data for
>>>>>>>
>>> applications
>>>
>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>> 5) generate JAPI results
>>>>>>>
>>>>>>> I'd like to fail this build at step 3, but I can't see an easy way
>>>>>>>
>>> to
>>>
>>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail if
>>>>>>>
>>> any
>>>
>>>>>>> of the module test sub-targets had test failures. I could parse
>>>>>>>
>>> the
>>>
>>>>>>> output I suppose, but I'd rather get ant to propagate the junit
>>>>>>>
>>> tasks
>>>
>>>>>>> failure property back up to the top level. I've tried a couple of
>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>
>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>>>
>>> create
>>>
>>>>> file "
>>>>>
>>>>>> TESTS.FAILED" in the root, when tests for some module fail. And in
>>>>>>
>>> the
>>>
>>>>> end
>>>>>
>>>>>> of tests suite run check whether this file exists on not.
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan.
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>>> Mark.
>>>>>>>
>>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I've checked out at morning last updates, built the code base
>>>>>>>>
>>> and
>>>
>>>>> run
>>>>>
>>>>>>> the
>>>>>>>
>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>
>>>>>>>> There are 9 tests failures in
>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest –
>>>>>>>>
>>> I
>>>
>>>>> saw
>>>>>
>>>>>>> these
>>>>>>>
>>>>>>>> failures before from time to time. It seems that tests depend on
>>>>>>>>
>>>>> some
>>>>>
>>>>>>> race
>>>>>>>
>>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>>>>>
>>> these
>>>
>>>>>>> tests
>>>>>>>
>>>>>>>> passing for you?
>>>>>>>>
>>>>>>>> And there are new 15 test failures. So now if I'll make a code
>>>>>>>>
>>>>> update
>>>>>
>>>>>>> or a
>>>>>>>
>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>>
>>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>>>
>>>>> notification
>>>>>
>>>>>>> with
>>>>>>>
>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
>>>>>>>>
>>> be
>>>
>>>>> send
>>>>>
>>>>>>> to
>>>>>>>
>>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>>>>>
>>> for
>>>
>>>>> tests?
>>>>>
>>>>>>> I
>>>>>>>
>>>>>>>> mean that after completing automatic build the existing classlib
>>>>>>>>
>>>>> tests
>>>>>
>>>>>>> suite
>>>>>>>
>>>>>>>> should be run. If there are failing tests then a report
>>>>>>>>
>>> notification
>>>
>>>>>>> with
>>>>>>>
>>>>>>>> corresponding subject will be send.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan Mishura
>>>>>>>> Intel Middleware Products Division
>>>>>>>>
>>>>>>> --
>>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>
>>>>>>>
>>>>>>>
>>> ---------------------------------------------------------------------
>>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail:
>>>>>>>
>>> harmony-dev-unsubscribe@incubator.apache.org
>>>
>>>>>>> For additional commands, e-mail:
>>>>>>>
>>> harmony-dev-help@incubator.apache.org
>>>
>>>>>>>
>>>>>> --
>>>>>>
>>>>>>
>>> ---------------------------------------------------------------------
>>>
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>>>>>
>>> harmony-dev-help@incubator.apache.org
>>>
>>>>>> Thanks,
>>>>>> Stepan Mishura
>>>>>> Intel Middleware Products Division
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> Mark Hindess <ma...@googlemail.com>
>>>>> IBM Java Technology Centre, UK.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>
>>>>>
>>>>>
>>>> --
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>> Thanks,
>>>> Stepan Mishura
>>>> Intel Middleware Products Division
>>>>
>>>>
>>>>
>>> --
>>> Mark Hindess <ma...@googlemail.com>
>>> IBM Java Technology Centre, UK.
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>>
>>
>>
>> --
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>> Thanks,
>> Stepan Mishura
>> Intel Middleware Products Division
>>
>>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by George Harley <ge...@googlemail.com>.
Hi,
It *seems* like things started failing after I committed the changes for
HARMONY-205 last night. I'm looking into this now. If the investigation
begins to take up too much time I will back the changes out.
Best regards,
George
Stepan Mishura wrote:
> The same for me + DatagramChannelTest
>
> Thanks,
> Stepan.
>
> On 4/11/06, Mark Hindess wrote:
>
>> No. These:
>>
>> F
>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
>> E
>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
>> E
>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>> E
>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>> F java.security.cert.X509CertSelectorTest.testMatch
>> F java.security.cert.X509CertSelectorTest.testClone
>> F tests.api.java.sql.DateTest.testSetTimelong
>> F tests.api.java.sql.DateTest.testToString
>> F tests.api.java.sql.DateTest.testValueOf
>> F tests.api.java.sql.TimestampTest.testSetNanosint
>> F tests.api.java.sql.TimestampTest.testToString
>> F tests.api.java.util.DateTest.test_toGMTString
>> F tests.api.java.util.DateTest.test_toString
>>
>> Regards,
>> Mark.
>>
>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>
>>> On 4/11/06, Mark Hindess wrote:
>>>
>>>> Yes. I was using the failureproperty mechanism. Trying to get this
>>>> property propogated back to the top level ant file was what I was
>>>> having trouble with.
>>>>
>>>> Using a file as you suggest might help. I'll give that a try
>>>>
>> shortly...
>>
>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>
>>> I guess that you have 9 tests from DatagramChannelTest passed. And 12 +
>>>
>> 3 =
>>
>>> 15 :-)
>>>
>>> (And
>>>
>>>> there are typos - "mathc" should be "match" - in the failure messages
>>>> for java.security.cert testMatch and testClone.)
>>>>
>>> I've fixed typo.
>>>
>>> Thanks,
>>> Stepan.
>>>
>>> Regards,
>>>
>>>> Mark.
>>>>
>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>
>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>
>>>>>
>>>>>> Stepan,
>>>>>>
>>>>>> I have another build running (but without notifications going to
>>>>>>
>> the
>>
>>>>>> list) that runs:
>>>>>>
>>>>>> 1) build (with reference jdk)
>>>>>> 2) build with what we created with 1)
>>>>>> 3) test
>>>>>> 4) create classlists and compare with class load data for
>>>>>>
>> applications
>>
>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>> 5) generate JAPI results
>>>>>>
>>>>>> I'd like to fail this build at step 3, but I can't see an easy way
>>>>>>
>> to
>>
>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail if
>>>>>>
>> any
>>
>>>>>> of the module test sub-targets had test failures. I could parse
>>>>>>
>> the
>>
>>>>>> output I suppose, but I'd rather get ant to propagate the junit
>>>>>>
>> tasks
>>
>>>>>> failure property back up to the top level. I've tried a couple of
>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>
>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>>
>> create
>>
>>>> file "
>>>>
>>>>> TESTS.FAILED" in the root, when tests for some module fail. And in
>>>>>
>> the
>>
>>>> end
>>>>
>>>>> of tests suite run check whether this file exists on not.
>>>>>
>>>>> Thanks,
>>>>> Stepan.
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>>> Mark.
>>>>>>
>>>>>> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I've checked out at morning last updates, built the code base
>>>>>>>
>> and
>>
>>>> run
>>>>
>>>>>> the
>>>>>>
>>>>>>> tests …and there are 24 tests failures!
>>>>>>>
>>>>>>> There are 9 tests failures in
>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest –
>>>>>>>
>> I
>>
>>>> saw
>>>>
>>>>>> these
>>>>>>
>>>>>>> failures before from time to time. It seems that tests depend on
>>>>>>>
>>>> some
>>>>
>>>>>> race
>>>>>>
>>>>>>> conditions because they may pass if I rerun them. Paulex, are
>>>>>>>
>> these
>>
>>>>>> tests
>>>>>>
>>>>>>> passing for you?
>>>>>>>
>>>>>>> And there are new 15 test failures. So now if I'll make a code
>>>>>>>
>>>> update
>>>>
>>>>>> or a
>>>>>>
>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>
>>>>>>> Currently if a commit breaks the Harmony classlib build a
>>>>>>>
>>>> notification
>>>>
>>>>>> with
>>>>>>
>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
>>>>>>>
>> be
>>
>>>> send
>>>>
>>>>>> to
>>>>>>
>>>>>>> harmony-commits mailing list. Is it possible to have the same
>>>>>>>
>> for
>>
>>>> tests?
>>>>
>>>>>> I
>>>>>>
>>>>>>> mean that after completing automatic build the existing classlib
>>>>>>>
>>>> tests
>>>>
>>>>>> suite
>>>>>>
>>>>>>> should be run. If there are failing tests then a report
>>>>>>>
>> notification
>>
>>>>>> with
>>>>>>
>>>>>>> corresponding subject will be send.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Stepan Mishura
>>>>>>> Intel Middleware Products Division
>>>>>>>
>>>>>> --
>>>>>> Mark Hindess <ma...@googlemail.com>
>>>>>> IBM Java Technology Centre, UK.
>>>>>>
>>>>>>
>>>>>>
>> ---------------------------------------------------------------------
>>
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail:
>>>>>>
>> harmony-dev-unsubscribe@incubator.apache.org
>>
>>>>>> For additional commands, e-mail:
>>>>>>
>> harmony-dev-help@incubator.apache.org
>>
>>>>>>
>>>>> --
>>>>>
>>>>>
>> ---------------------------------------------------------------------
>>
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail:
>>>>>
>> harmony-dev-help@incubator.apache.org
>>
>>>>> Thanks,
>>>>> Stepan Mishura
>>>>> Intel Middleware Products Division
>>>>>
>>>>>
>>>>>
>>>> --
>>>> Mark Hindess <ma...@googlemail.com>
>>>> IBM Java Technology Centre, UK.
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>>
>>> --
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>> Thanks,
>>> Stepan Mishura
>>> Intel Middleware Products Division
>>>
>>>
>>>
>> --
>> Mark Hindess <ma...@googlemail.com>
>> IBM Java Technology Centre, UK.
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>>
>
>
> --
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Stepan Mishura <st...@gmail.com>.
The same for me + DatagramChannelTest
Thanks,
Stepan.
On 4/11/06, Mark Hindess wrote:
>
> No. These:
>
> F
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
> E
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
> E
> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
> E
> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
> F java.security.cert.X509CertSelectorTest.testMatch
> F java.security.cert.X509CertSelectorTest.testClone
> F tests.api.java.sql.DateTest.testSetTimelong
> F tests.api.java.sql.DateTest.testToString
> F tests.api.java.sql.DateTest.testValueOf
> F tests.api.java.sql.TimestampTest.testSetNanosint
> F tests.api.java.sql.TimestampTest.testToString
> F tests.api.java.util.DateTest.test_toGMTString
> F tests.api.java.util.DateTest.test_toString
>
> Regards,
> Mark.
>
> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > On 4/11/06, Mark Hindess wrote:
> > >
> > > Yes. I was using the failureproperty mechanism. Trying to get this
> > > property propogated back to the top level ant file was what I was
> > > having trouble with.
> > >
> > > Using a file as you suggest might help. I'll give that a try
> shortly...
> > >
> > > Incidentally, I'm seeing 12 failures and 3 errors on r393111.
> >
> >
> > I guess that you have 9 tests from DatagramChannelTest passed. And 12 +
> 3 =
> > 15 :-)
> >
> > (And
> > > there are typos - "mathc" should be "match" - in the failure messages
> > > for java.security.cert testMatch and testClone.)
> >
> >
> > I've fixed typo.
> >
> > Thanks,
> > Stepan.
> >
> > Regards,
> > > Mark.
> > >
> > > On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > > On 4/11/06, Mark Hindess wrote:
> > > >
> > > > > Stepan,
> > > > >
> > > > > I have another build running (but without notifications going to
> the
> > > > > list) that runs:
> > > > >
> > > > > 1) build (with reference jdk)
> > > > > 2) build with what we created with 1)
> > > > > 3) test
> > > > > 4) create classlists and compare with class load data for
> applications
> > > > > (tomcat, geronimo, continuum, etc.)
> > > > > 5) generate JAPI results
> > > > >
> > > > > I'd like to fail this build at step 3, but I can't see an easy way
> to
> > > > > get 'ant -f make/build.xml test' to run all tests and then fail if
> any
> > > > > of the module test sub-targets had test failures. I could parse
> the
> > > > > output I suppose, but I'd rather get ant to propagate the junit
> tasks
> > > > > failure property back up to the top level. I've tried a couple of
> > > > > things and none seem to work. Any suggestions welcome.
> > > >
> > > >
> > > > Mark, did you try failureproperty parameter for junit task?
> > > > We may add it to ant sub-targets to raise a flag, for example,
> create
> > > file "
> > > > TESTS.FAILED" in the root, when tests for some module fail. And in
> the
> > > end
> > > > of tests suite run check whether this file exists on not.
> > > >
> > > > Thanks,
> > > > Stepan.
> > > >
> > > >
> > > > Regards,
> > > > > Mark.
> > > > >
> > > > > On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > > > > Hi,
> > > > > >
> > > > > > I've checked out at morning last updates, built the code base
> and
> > > run
> > > > > the
> > > > > > tests …and there are 24 tests failures!
> > > > > >
> > > > > > There are 9 tests failures in
> > > > > > org.apache.harmony.tests.java.nio.channels.DatagramChannelTest –
> I
> > > saw
> > > > > these
> > > > > > failures before from time to time. It seems that tests depend on
> > > some
> > > > > race
> > > > > > conditions because they may pass if I rerun them. Paulex, are
> these
> > > > > tests
> > > > > > passing for you?
> > > > > >
> > > > > > And there are new 15 test failures. So now if I'll make a code
> > > update
> > > > > or a
> > > > > > bug fix how I can be 100% sure that I don't do any regression?
> > > > > >
> > > > > > Currently if a commit breaks the Harmony classlib build a
> > > notification
> > > > > with
> > > > > > subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will
> be
> > > send
> > > > > to
> > > > > > harmony-commits mailing list. Is it possible to have the same
> for
> > > tests?
> > > > > I
> > > > > > mean that after completing automatic build the existing classlib
> > > tests
> > > > > suite
> > > > > > should be run. If there are failing tests then a report
> notification
> > > > > with
> > > > > > corresponding subject will be send.
> > > > > >
> > > > > > Thanks,
> > > > > > Stepan Mishura
> > > > > > Intel Middleware Products Division
> > > > >
> > > > > --
> > > > > Mark Hindess <ma...@googlemail.com>
> > > > > IBM Java Technology Centre, UK.
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > > > > For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > > >
> > > > Thanks,
> > > > Stepan Mishura
> > > > Intel Middleware Products Division
> > > >
> > > >
> > >
> > >
> > > --
> > > Mark Hindess <ma...@googlemail.com>
> > > IBM Java Technology Centre, UK.
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
> > --
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> > Thanks,
> > Stepan Mishura
> > Intel Middleware Products Division
> >
> >
>
>
> --
> Mark Hindess <ma...@googlemail.com>
> IBM Java Technology Centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Thanks,
Stepan Mishura
Intel Middleware Products Division
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
No. These:
F org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
E org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
E org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
E org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
F java.security.cert.X509CertSelectorTest.testMatch
F java.security.cert.X509CertSelectorTest.testClone
F tests.api.java.sql.DateTest.testSetTimelong
F tests.api.java.sql.DateTest.testToString
F tests.api.java.sql.DateTest.testValueOf
F tests.api.java.sql.TimestampTest.testSetNanosint
F tests.api.java.sql.TimestampTest.testToString
F tests.api.java.util.DateTest.test_toGMTString
F tests.api.java.util.DateTest.test_toString
Regards,
Mark.
On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> On 4/11/06, Mark Hindess wrote:
> >
> > Yes. I was using the failureproperty mechanism. Trying to get this
> > property propogated back to the top level ant file was what I was
> > having trouble with.
> >
> > Using a file as you suggest might help. I'll give that a try shortly...
> >
> > Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>
>
> I guess that you have 9 tests from DatagramChannelTest passed. And 12 + 3 =
> 15 :-)
>
> (And
> > there are typos - "mathc" should be "match" - in the failure messages
> > for java.security.cert testMatch and testClone.)
>
>
> I've fixed typo.
>
> Thanks,
> Stepan.
>
> Regards,
> > Mark.
> >
> > On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > On 4/11/06, Mark Hindess wrote:
> > >
> > > > Stepan,
> > > >
> > > > I have another build running (but without notifications going to the
> > > > list) that runs:
> > > >
> > > > 1) build (with reference jdk)
> > > > 2) build with what we created with 1)
> > > > 3) test
> > > > 4) create classlists and compare with class load data for applications
> > > > (tomcat, geronimo, continuum, etc.)
> > > > 5) generate JAPI results
> > > >
> > > > I'd like to fail this build at step 3, but I can't see an easy way to
> > > > get 'ant -f make/build.xml test' to run all tests and then fail if any
> > > > of the module test sub-targets had test failures. I could parse the
> > > > output I suppose, but I'd rather get ant to propagate the junit tasks
> > > > failure property back up to the top level. I've tried a couple of
> > > > things and none seem to work. Any suggestions welcome.
> > >
> > >
> > > Mark, did you try failureproperty parameter for junit task?
> > > We may add it to ant sub-targets to raise a flag, for example, create
> > file "
> > > TESTS.FAILED" in the root, when tests for some module fail. And in the
> > end
> > > of tests suite run check whether this file exists on not.
> > >
> > > Thanks,
> > > Stepan.
> > >
> > >
> > > Regards,
> > > > Mark.
> > > >
> > > > On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > > > Hi,
> > > > >
> > > > > I've checked out at morning last updates, built the code base and
> > run
> > > > the
> > > > > tests …and there are 24 tests failures!
> > > > >
> > > > > There are 9 tests failures in
> > > > > org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I
> > saw
> > > > these
> > > > > failures before from time to time. It seems that tests depend on
> > some
> > > > race
> > > > > conditions because they may pass if I rerun them. Paulex, are these
> > > > tests
> > > > > passing for you?
> > > > >
> > > > > And there are new 15 test failures. So now if I'll make a code
> > update
> > > > or a
> > > > > bug fix how I can be 100% sure that I don't do any regression?
> > > > >
> > > > > Currently if a commit breaks the Harmony classlib build a
> > notification
> > > > with
> > > > > subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be
> > send
> > > > to
> > > > > harmony-commits mailing list. Is it possible to have the same for
> > tests?
> > > > I
> > > > > mean that after completing automatic build the existing classlib
> > tests
> > > > suite
> > > > > should be run. If there are failing tests then a report notification
> > > > with
> > > > > corresponding subject will be send.
> > > > >
> > > > > Thanks,
> > > > > Stepan Mishura
> > > > > Intel Middleware Products Division
> > > >
> > > > --
> > > > Mark Hindess <ma...@googlemail.com>
> > > > IBM Java Technology Centre, UK.
> > > >
> > > > ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > >
> > > --
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > > Thanks,
> > > Stepan Mishura
> > > Intel Middleware Products Division
> > >
> > >
> >
> >
> > --
> > Mark Hindess <ma...@googlemail.com>
> > IBM Java Technology Centre, UK.
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> --
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
>
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Stepan Mishura <st...@gmail.com>.
On 4/11/06, Mark Hindess wrote:
>
> Yes. I was using the failureproperty mechanism. Trying to get this
> property propogated back to the top level ant file was what I was
> having trouble with.
>
> Using a file as you suggest might help. I'll give that a try shortly...
>
> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
I guess that you have 9 tests from DatagramChannelTest passed. And 12 + 3 =
15 :-)
(And
> there are typos - "mathc" should be "match" - in the failure messages
> for java.security.cert testMatch and testClone.)
I've fixed typo.
Thanks,
Stepan.
Regards,
> Mark.
>
> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > On 4/11/06, Mark Hindess wrote:
> >
> > > Stepan,
> > >
> > > I have another build running (but without notifications going to the
> > > list) that runs:
> > >
> > > 1) build (with reference jdk)
> > > 2) build with what we created with 1)
> > > 3) test
> > > 4) create classlists and compare with class load data for applications
> > > (tomcat, geronimo, continuum, etc.)
> > > 5) generate JAPI results
> > >
> > > I'd like to fail this build at step 3, but I can't see an easy way to
> > > get 'ant -f make/build.xml test' to run all tests and then fail if any
> > > of the module test sub-targets had test failures. I could parse the
> > > output I suppose, but I'd rather get ant to propagate the junit tasks
> > > failure property back up to the top level. I've tried a couple of
> > > things and none seem to work. Any suggestions welcome.
> >
> >
> > Mark, did you try failureproperty parameter for junit task?
> > We may add it to ant sub-targets to raise a flag, for example, create
> file "
> > TESTS.FAILED" in the root, when tests for some module fail. And in the
> end
> > of tests suite run check whether this file exists on not.
> >
> > Thanks,
> > Stepan.
> >
> >
> > Regards,
> > > Mark.
> > >
> > > On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > > Hi,
> > > >
> > > > I've checked out at morning last updates, built the code base and
> run
> > > the
> > > > tests …and there are 24 tests failures!
> > > >
> > > > There are 9 tests failures in
> > > > org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I
> saw
> > > these
> > > > failures before from time to time. It seems that tests depend on
> some
> > > race
> > > > conditions because they may pass if I rerun them. Paulex, are these
> > > tests
> > > > passing for you?
> > > >
> > > > And there are new 15 test failures. So now if I'll make a code
> update
> > > or a
> > > > bug fix how I can be 100% sure that I don't do any regression?
> > > >
> > > > Currently if a commit breaks the Harmony classlib build a
> notification
> > > with
> > > > subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be
> send
> > > to
> > > > harmony-commits mailing list. Is it possible to have the same for
> tests?
> > > I
> > > > mean that after completing automatic build the existing classlib
> tests
> > > suite
> > > > should be run. If there are failing tests then a report notification
> > > with
> > > > corresponding subject will be send.
> > > >
> > > > Thanks,
> > > > Stepan Mishura
> > > > Intel Middleware Products Division
> > >
> > > --
> > > Mark Hindess <ma...@googlemail.com>
> > > IBM Java Technology Centre, UK.
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
> > --
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> > Thanks,
> > Stepan Mishura
> > Intel Middleware Products Division
> >
> >
>
>
> --
> Mark Hindess <ma...@googlemail.com>
> IBM Java Technology Centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Thanks,
Stepan Mishura
Intel Middleware Products Division
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
Yes. I was using the failureproperty mechanism. Trying to get this
property propogated back to the top level ant file was what I was
having trouble with.
Using a file as you suggest might help. I'll give that a try shortly...
Incidentally, I'm seeing 12 failures and 3 errors on r393111. (And
there are typos - "mathc" should be "match" - in the failure messages
for java.security.cert testMatch and testClone.)
Regards,
Mark.
On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> On 4/11/06, Mark Hindess wrote:
>
> > Stepan,
> >
> > I have another build running (but without notifications going to the
> > list) that runs:
> >
> > 1) build (with reference jdk)
> > 2) build with what we created with 1)
> > 3) test
> > 4) create classlists and compare with class load data for applications
> > (tomcat, geronimo, continuum, etc.)
> > 5) generate JAPI results
> >
> > I'd like to fail this build at step 3, but I can't see an easy way to
> > get 'ant -f make/build.xml test' to run all tests and then fail if any
> > of the module test sub-targets had test failures. I could parse the
> > output I suppose, but I'd rather get ant to propagate the junit tasks
> > failure property back up to the top level. I've tried a couple of
> > things and none seem to work. Any suggestions welcome.
>
>
> Mark, did you try failureproperty parameter for junit task?
> We may add it to ant sub-targets to raise a flag, for example, create file "
> TESTS.FAILED" in the root, when tests for some module fail. And in the end
> of tests suite run check whether this file exists on not.
>
> Thanks,
> Stepan.
>
>
> Regards,
> > Mark.
> >
> > On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > > Hi,
> > >
> > > I've checked out at morning last updates, built the code base and run
> > the
> > > tests …and there are 24 tests failures!
> > >
> > > There are 9 tests failures in
> > > org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I saw
> > these
> > > failures before from time to time. It seems that tests depend on some
> > race
> > > conditions because they may pass if I rerun them. Paulex, are these
> > tests
> > > passing for you?
> > >
> > > And there are new 15 test failures. So now if I'll make a code update
> > or a
> > > bug fix how I can be 100% sure that I don't do any regression?
> > >
> > > Currently if a commit breaks the Harmony classlib build a notification
> > with
> > > subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be send
> > to
> > > harmony-commits mailing list. Is it possible to have the same for tests?
> > I
> > > mean that after completing automatic build the existing classlib tests
> > suite
> > > should be run. If there are failing tests then a report notification
> > with
> > > corresponding subject will be send.
> > >
> > > Thanks,
> > > Stepan Mishura
> > > Intel Middleware Products Division
> >
> > --
> > Mark Hindess <ma...@googlemail.com>
> > IBM Java Technology Centre, UK.
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> --
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
>
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Stepan Mishura <st...@gmail.com>.
On 4/11/06, Mark Hindess wrote:
> Stepan,
>
> I have another build running (but without notifications going to the
> list) that runs:
>
> 1) build (with reference jdk)
> 2) build with what we created with 1)
> 3) test
> 4) create classlists and compare with class load data for applications
> (tomcat, geronimo, continuum, etc.)
> 5) generate JAPI results
>
> I'd like to fail this build at step 3, but I can't see an easy way to
> get 'ant -f make/build.xml test' to run all tests and then fail if any
> of the module test sub-targets had test failures. I could parse the
> output I suppose, but I'd rather get ant to propagate the junit tasks
> failure property back up to the top level. I've tried a couple of
> things and none seem to work. Any suggestions welcome.
Mark, did you try failureproperty parameter for junit task?
We may add it to ant sub-targets to raise a flag, for example, create file "
TESTS.FAILED" in the root, when tests for some module fail. And in the end
of tests suite run check whether this file exists on not.
Thanks,
Stepan.
Regards,
> Mark.
>
> On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> > Hi,
> >
> > I've checked out at morning last updates, built the code base and run
> the
> > tests …and there are 24 tests failures!
> >
> > There are 9 tests failures in
> > org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I saw
> these
> > failures before from time to time. It seems that tests depend on some
> race
> > conditions because they may pass if I rerun them. Paulex, are these
> tests
> > passing for you?
> >
> > And there are new 15 test failures. So now if I'll make a code update
> or a
> > bug fix how I can be 100% sure that I don't do any regression?
> >
> > Currently if a commit breaks the Harmony classlib build a notification
> with
> > subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be send
> to
> > harmony-commits mailing list. Is it possible to have the same for tests?
> I
> > mean that after completing automatic build the existing classlib tests
> suite
> > should be run. If there are failing tests then a report notification
> with
> > corresponding subject will be send.
> >
> > Thanks,
> > Stepan Mishura
> > Intel Middleware Products Division
>
> --
> Mark Hindess <ma...@googlemail.com>
> IBM Java Technology Centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Thanks,
Stepan Mishura
Intel Middleware Products Division
Re: classlib test suite status emails?
Posted by Mark Hindess <ma...@googlemail.com>.
Stepan,
I have another build running (but without notifications going to the
list) that runs:
1) build (with reference jdk)
2) build with what we created with 1)
3) test
4) create classlists and compare with class load data for applications
(tomcat, geronimo, continuum, etc.)
5) generate JAPI results
I'd like to fail this build at step 3, but I can't see an easy way to
get 'ant -f make/build.xml test' to run all tests and then fail if any
of the module test sub-targets had test failures. I could parse the
output I suppose, but I'd rather get ant to propagate the junit tasks
failure property back up to the top level. I've tried a couple of
things and none seem to work. Any suggestions welcome.
Regards,
Mark.
On 4/11/06, Stepan Mishura <st...@gmail.com> wrote:
> Hi,
>
> I've checked out at morning last updates, built the code base and run the
> tests …and there are 24 tests failures!
>
> There are 9 tests failures in
> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I saw these
> failures before from time to time. It seems that tests depend on some race
> conditions because they may pass if I rerun them. Paulex, are these tests
> passing for you?
>
> And there are new 15 test failures. So now if I'll make a code update or a
> bug fix how I can be 100% sure that I don't do any regression?
>
> Currently if a commit breaks the Harmony classlib build a notification with
> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be send to
> harmony-commits mailing list. Is it possible to have the same for tests? I
> mean that after completing automatic build the existing classlib tests suite
> should be run. If there are failing tests then a report notification with
> corresponding subject will be send.
>
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
--
Mark Hindess <ma...@googlemail.com>
IBM Java Technology Centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by LvJimmy，Jing <fi...@gmail.com>.
So it really happens... platform dependency, or platform-release dependenecy
make things in mess.
In my opinion, George Harley's method of grouping test above and in the
threads of "matching reference implementation exception behaviour" is fairly
good.
And another serious problem leaves here: how can we know, the test is
platfrom dependency? Yes, some of them are obvious, but the other, just like
the problems here, are not.
2006/4/11, Paulex Yang <pa...@gmail.com>:
>
> Stepan Mishura wrote:
> > On 4/11/06, Paulex Yang wrote:
> >
> >
> >> Stepan Mishura wrote:
> >>
> >>> Hi,
> >>>
> >>> I've checked out at morning last updates, built the code base and run
> >>>
> >> the
> >>
> >>> tests ��and there are 24 tests failures!
> >>>
> >>> There are 9 tests failures in
> >>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest �C I saw
> >>>
> >> these
> >>
> >>> failures before from time to time. It seems that tests depend on some
> >>>
> >> race
> >>
> >>> conditions because they may pass if I rerun them. Paulex, are these
> >>>
> >> tests
> >>
> >>> passing for you?
> >>>
> >>>
> >>>
> >> Mishura
> >>
> >> the channels test cases on my workstations(yes I tried on several
> >> machines) always pass, but it is still possible that some of them are
> >> unstable and depend on race condition for others, I might miss to catch
> >> them by mistake. Would you pls. give me more details about the failure?
>
> >> I'll try to investigate them further. thank you.
> >>
> >
> >
> > I've run tests on Linux. They fail on the same assertion:
> > [junit] Testcase: testReceiveSend_Block_Normal(
> > org.apache.harmony.tests.java.nio.channels.DatagramChannelTest):
> FAILED
> > [junit] expected:<...> but was:<....localdomain>
> > [junit] junit.framework.ComparisonFailure: expected:<...> but
> > was:<....localdomain>
> > [junit] at
> >
> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest.receiveByChannel
> > (DatagramChannelTest.java:1681)
> > [junit] at
> >
> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest.testReceiveSend_Block_Normal
> > (DatagramChannelTest.java:1419)
> > [junit] at java.lang.reflect.AccessibleObject.invokeV(
> > AccessibleObject.java :205)
> >
> Stepan,
>
> Thank you for the information.
> > As far as JUnit didn't print what result was expected to get I inserted
> > printing to the test and got the following:
> > Expected host: localhost.localdomain
> > Recieved host: localhost
> >
> > Why do you expect that on Linux "localhost.localdomain" should be
> returned?
> >
> Well, that is because both RI/Harmony return this value on my test
> machines(Redhat ES 3, Redhat 9 and latest stable Debian), and I didn't
> expect this value is dependent on configuration(or Linux distribution?
> ), but as your test result shows, it probably does! So would you tell me
> your linux distro version and configuration( for example, what it looks
> like of the item on localhost in /etc/hosts)? so that I can check if
> this constant value(localhost.localdomain) is platform dependent.
>
> And I'll also look more inside to see if there can be other cause except
> for the possible platform difference.
> > Thanks,
> > Stepan.
> >
> >
> >
> >> And before everything's OK, I'm fine if you choose to add
> >> DatagramChannelTest into exclude list, by adding it to Ln.94 of
> >> modules/nio/make/common/build.xml, just as ServerSocketChannelTest.
> >>
> >> Sorry for the inconvenience caused.
> >>
> >>> And there are new 15 test failures. So now if I'll make a code update
>
> >>>
> >> or a
> >>
> >>> bug fix how I can be 100% sure that I don't do any regression?
> >>>
> >>> Currently if a commit breaks the Harmony classlib build a notification
>
> >>>
> >> with
> >>
> >>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be send
> >>>
> >> to
> >>
> >>> harmony-commits mailing list. Is it possible to have the same for
> tests?
> >>>
> >> I
> >>
> >>> mean that after completing automatic build the existing classlib tests
> >>>
> >> suite
> >>
> >>> should be run. If there are failing tests then a report notification
> >>>
> >> with
> >>
> >>> corresponding subject will be send.
> >>>
> >>> Thanks,
> >>> Stepan Mishura
> >>> Intel Middleware Products Division
> >>> ---------------------------------------------------------------------
> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>
> >>>
> >>>
> >> --
> >> Paulex Yang
> >> China Software Development Lab
> >> IBM
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >>
> >>
> >
> >
> > --
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> > Thanks,
> > Stepan Mishura
> > Intel Middleware Products Division
> >
> >
>
>
> --
> Paulex Yang
> China Software Development Lab
> IBM
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Best Regards!
Jimmy, Jing Lv
China Software Development Lab, IBM
Re: classlib test suite status emails?
Posted by Paulex Yang <pa...@gmail.com>.
Stepan Mishura wrote:
> On 4/11/06, Paulex Yang wrote:
>
>
>> Stepan Mishura wrote:
>>
>>> Hi,
>>>
>>> I've checked out at morning last updates, built the code base and run
>>>
>> the
>>
>>> tests …and there are 24 tests failures!
>>>
>>> There are 9 tests failures in
>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I saw
>>>
>> these
>>
>>> failures before from time to time. It seems that tests depend on some
>>>
>> race
>>
>>> conditions because they may pass if I rerun them. Paulex, are these
>>>
>> tests
>>
>>> passing for you?
>>>
>>>
>>>
>> Mishura
>>
>> the channels test cases on my workstations(yes I tried on several
>> machines) always pass, but it is still possible that some of them are
>> unstable and depend on race condition for others, I might miss to catch
>> them by mistake. Would you pls. give me more details about the failure?
>> I'll try to investigate them further. thank you.
>>
>
>
> I've run tests on Linux. They fail on the same assertion:
> [junit] Testcase: testReceiveSend_Block_Normal(
> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest): FAILED
> [junit] expected:<...> but was:<....localdomain>
> [junit] junit.framework.ComparisonFailure: expected:<...> but
> was:<....localdomain>
> [junit] at
> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest.receiveByChannel
> (DatagramChannelTest.java:1681)
> [junit] at
> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest.testReceiveSend_Block_Normal
> (DatagramChannelTest.java:1419)
> [junit] at java.lang.reflect.AccessibleObject.invokeV(
> AccessibleObject.java:205)
>
Stepan,
Thank you for the information.
> As far as JUnit didn't print what result was expected to get I inserted
> printing to the test and got the following:
> Expected host: localhost.localdomain
> Recieved host: localhost
>
> Why do you expect that on Linux "localhost.localdomain" should be returned?
>
Well, that is because both RI/Harmony return this value on my test
machines(Redhat ES 3, Redhat 9 and latest stable Debian), and I didn't
expect this value is dependent on configuration(or Linux distribution?
), but as your test result shows, it probably does! So would you tell me
your linux distro version and configuration( for example, what it looks
like of the item on localhost in /etc/hosts)? so that I can check if
this constant value(localhost.localdomain) is platform dependent.
And I'll also look more inside to see if there can be other cause except
for the possible platform difference.
> Thanks,
> Stepan.
>
>
>
>> And before everything's OK, I'm fine if you choose to add
>> DatagramChannelTest into exclude list, by adding it to Ln.94 of
>> modules/nio/make/common/build.xml, just as ServerSocketChannelTest.
>>
>> Sorry for the inconvenience caused.
>>
>>> And there are new 15 test failures. So now if I'll make a code update
>>>
>> or a
>>
>>> bug fix how I can be 100% sure that I don't do any regression?
>>>
>>> Currently if a commit breaks the Harmony classlib build a notification
>>>
>> with
>>
>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be send
>>>
>> to
>>
>>> harmony-commits mailing list. Is it possible to have the same for tests?
>>>
>> I
>>
>>> mean that after completing automatic build the existing classlib tests
>>>
>> suite
>>
>>> should be run. If there are failing tests then a report notification
>>>
>> with
>>
>>> corresponding subject will be send.
>>>
>>> Thanks,
>>> Stepan Mishura
>>> Intel Middleware Products Division
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>>
>> --
>> Paulex Yang
>> China Software Development Lab
>> IBM
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>>
>
>
> --
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
>
--
Paulex Yang
China Software Development Lab
IBM
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: classlib test suite status emails?
Posted by Stepan Mishura <st...@gmail.com>.
On 4/11/06, Paulex Yang wrote:
> Stepan Mishura wrote:
> > Hi,
> >
> > I've checked out at morning last updates, built the code base and run
> the
> > tests …and there are 24 tests failures!
> >
> > There are 9 tests failures in
> > org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I saw
> these
> > failures before from time to time. It seems that tests depend on some
> race
> > conditions because they may pass if I rerun them. Paulex, are these
> tests
> > passing for you?
> >
> >
> Mishura
>
> the channels test cases on my workstations(yes I tried on several
> machines) always pass, but it is still possible that some of them are
> unstable and depend on race condition for others, I might miss to catch
> them by mistake. Would you pls. give me more details about the failure?
> I'll try to investigate them further. thank you.
I've run tests on Linux. They fail on the same assertion:
[junit] Testcase: testReceiveSend_Block_Normal(
org.apache.harmony.tests.java.nio.channels.DatagramChannelTest): FAILED
[junit] expected:<...> but was:<....localdomain>
[junit] junit.framework.ComparisonFailure: expected:<...> but
was:<....localdomain>
[junit] at
org.apache.harmony.tests.java.nio.channels.DatagramChannelTest.receiveByChannel
(DatagramChannelTest.java:1681)
[junit] at
org.apache.harmony.tests.java.nio.channels.DatagramChannelTest.testReceiveSend_Block_Normal
(DatagramChannelTest.java:1419)
[junit] at java.lang.reflect.AccessibleObject.invokeV(
AccessibleObject.java:205)
As far as JUnit didn't print what result was expected to get I inserted
printing to the test and got the following:
Expected host: localhost.localdomain
Recieved host: localhost
Why do you expect that on Linux "localhost.localdomain" should be returned?
Thanks,
Stepan.
> And before everything's OK, I'm fine if you choose to add
> DatagramChannelTest into exclude list, by adding it to Ln.94 of
> modules/nio/make/common/build.xml, just as ServerSocketChannelTest.
>
> Sorry for the inconvenience caused.
> > And there are new 15 test failures. So now if I'll make a code update
> or a
> > bug fix how I can be 100% sure that I don't do any regression?
> >
> > Currently if a commit breaks the Harmony classlib build a notification
> with
> > subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be send
> to
> > harmony-commits mailing list. Is it possible to have the same for tests?
> I
> > mean that after completing automatic build the existing classlib tests
> suite
> > should be run. If there are failing tests then a report notification
> with
> > corresponding subject will be send.
> >
> > Thanks,
> > Stepan Mishura
> > Intel Middleware Products Division
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> --
> Paulex Yang
> China Software Development Lab
> IBM
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Thanks,
Stepan Mishura
Intel Middleware Products Division
Re: classlib test suite status emails?
Posted by Paulex Yang <pa...@gmail.com>.
Stepan Mishura wrote:
> Hi,
>
> I've checked out at morning last updates, built the code base and run the
> tests …and there are 24 tests failures!
>
> There are 9 tests failures in
> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest – I saw these
> failures before from time to time. It seems that tests depend on some race
> conditions because they may pass if I rerun them. Paulex, are these tests
> passing for you?
>
>
Mishura
the channels test cases on my workstations(yes I tried on several
machines) always pass, but it is still possible that some of them are
unstable and depend on race condition for others, I might miss to catch
them by mistake. Would you pls. give me more details about the failure?
I'll try to investigate them further. thank you.
And before everything's OK, I'm fine if you choose to add
DatagramChannelTest into exclude list, by adding it to Ln.94 of
modules/nio/make/common/build.xml, just as ServerSocketChannelTest.
Sorry for the inconvenience caused.
> And there are new 15 test failures. So now if I'll make a code update or a
> bug fix how I can be 100% sure that I don't do any regression?
>
> Currently if a commit breaks the Harmony classlib build a notification with
> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will be send to
> harmony-commits mailing list. Is it possible to have the same for tests? I
> mean that after completing automatic build the existing classlib tests suite
> should be run. If there are failing tests then a report notification with
> corresponding subject will be send.
>
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Paulex Yang
China Software Development Lab
IBM
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org