You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Leo Li <li...@gmail.com> on 2006/10/17 03:00:23 UTC
[classlib]Harmony passes 73% on Derby.
Hi, all:
I have ran the tests of Derby on Harmony, both on windows xp and
ubuntu. The result is similar.
On windows,
579 Tests Run
73% Pass (428 tests passed)
27% Fail (151 tests failed)
9 Suites skipped
On linux,
579 Tests Run
74% Pass (430 tests passed)
26% Fail (149 tests failed)
9 Suites skipped
The result seems a little frustrating since Sian had 82% passed several
months before. The version of derby I ran is at revision 464429. The lost
passing rate may be due to the modified code and added testcases.
To Sian: I made a new patch in the attachment which is an updated
version of the patch Sian provides. I will be appreciate if Sian helps to
check its validity and thus I will be assured that the lost passing rate is
irrelevant to our modification.
When I can confirm everything is all right, I will then put the ongoing
progress result on Harmony's wiki page.
Thank you.
--
Leo Li
China Software Development Lab, IBM
Re: [classlib]Harmony passes 73% on Derby.
Posted by Mikhail Fursov <mi...@gmail.com>.
Oh, sorry. I read this thread, it mentions the harmonyvm, the stability
degradation in the last months so the first thought was about DRLVM and
recent BBC affect :)
On 10/17/06, Leo Li <li...@gmail.com> wrote:
>
> Mikhail, unfortunately derby test cannot run on drlvm.
>
> An assert fail exception will be thrown out.
>
> I have talked about it in another thread
> http://www.mail-archive.com/harmony-dev@incubator.apache.org/msg15643.html
> and raised jira as 1836.
>
> On 10/17/06, Mikhail Fursov <mi...@gmail.com> wrote:
> >
> > Leo, just a simple interest, could you try to run harmony with -Xem:opt,
> > -Xem:jet, -Xem:server, -Xem:server_static options? These are all modes
> > drlvm
> > supports today except the default one (-Xem:client)
> >
> > For example if the test passes with Jitrino.JET (-Xem:jet) and fails
> with
> > Jitrino.OPT (-Xem:opt) this is a good candidate for JIT bug
> >
> > On 10/17/06, Leo Li <li...@gmail.com> wrote:
> > >
> > > Hi,
> > > The result of Derby is not so obvious, since its test is not junit
> > but
> > > some a serial of stand-alone java program calling each other. And the
> > > comparing result is a pre-stored data file. So I have to use sometime
> to
> > > set
> > > aside all the irrelevant code to spot those related with Harmony
> > classlib.
> > > What makes things worse is that as a product, all the exception
> has
> > > been
> > > caught...
> > > Is there someone who has good idea to pick up problem quickly in
> > this
> > > issue?
> > >
> > >
> > > On 10/17/06, Nathan Beyer <nb...@gmail.com> wrote:
> > > >
> > > > Are there any obvious bugs in Harmony that are of note? Perhaps some
> > > > unexpected NPEs?
> > > >
> > > > On 10/16/06, Leo Li <li...@gmail.com> wrote:
> > > > > Hi, all:
> > > > > I have ran the tests of Derby on Harmony, both on windows xp
> > and
> > > > > ubuntu. The result is similar.
> > > > >
> > > > > On windows,
> > > > > 579 Tests Run
> > > > > 73% Pass (428 tests passed)
> > > > > 27% Fail (151 tests failed)
> > > > > 9 Suites skipped
> > > > > On linux,
> > > > > 579 Tests Run
> > > > > 74% Pass (430 tests passed)
> > > > > 26% Fail (149 tests failed)
> > > > > 9 Suites skipped
> > > > >
> > > > > The result seems a little frustrating since Sian had 82%
> passed
> > > > several
> > > > > months before. The version of derby I ran is at revision 464429.
> The
> > > > lost
> > > > > passing rate may be due to the modified code and added testcases.
> > > > >
> > > > > To Sian: I made a new patch in the attachment which is an
> > updated
> > > > > version of the patch Sian provides. I will be appreciate if Sian
> > helps
> > > > to
> > > > > check its validity and thus I will be assured that the lost
> passing
> > > rate
> > > > is
> > > > > irrelevant to our modification.
> > > > >
> > > > > When I can confirm everything is all right, I will then put
> the
> > > > ongoing
> > > > > progress result on Harmony's wiki page.
> > > > >
> > > > > Thank you.
> > > > >
> > > > > --
> > > > > Leo Li
> > > > > China Software Development Lab, IBM
> > > > >
> > ---------------------------------------------------------------------
> > > > > Terms of use :
> > > > > http://incubator.apache.org/harmony/mailing.html
> > > > > To unsubscribe, e-mail:
> > > > > harmony-dev-unsubscribe@incubator.apache.org
> > > > > For additional commands, e-mail:
> > > > > harmony-dev-help@incubator.apache.org
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > >
> > > --
> > > Leo Li
> > > China Software Development Lab, IBM
> > >
> > >
> >
> >
> > --
> > Mikhail Fursov
> >
> >
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>
--
Mikhail Fursov
Re: [classlib]Harmony passes 73% on Derby.
Posted by Leo Li <li...@gmail.com>.
Mikhail, unfortunately derby test cannot run on drlvm.
An assert fail exception will be thrown out.
I have talked about it in another thread
http://www.mail-archive.com/harmony-dev@incubator.apache.org/msg15643.html
and raised jira as 1836.
On 10/17/06, Mikhail Fursov <mi...@gmail.com> wrote:
>
> Leo, just a simple interest, could you try to run harmony with -Xem:opt,
> -Xem:jet, -Xem:server, -Xem:server_static options? These are all modes
> drlvm
> supports today except the default one (-Xem:client)
>
> For example if the test passes with Jitrino.JET (-Xem:jet) and fails with
> Jitrino.OPT (-Xem:opt) this is a good candidate for JIT bug
>
> On 10/17/06, Leo Li <li...@gmail.com> wrote:
> >
> > Hi,
> > The result of Derby is not so obvious, since its test is not junit
> but
> > some a serial of stand-alone java program calling each other. And the
> > comparing result is a pre-stored data file. So I have to use sometime to
> > set
> > aside all the irrelevant code to spot those related with Harmony
> classlib.
> > What makes things worse is that as a product, all the exception has
> > been
> > caught...
> > Is there someone who has good idea to pick up problem quickly in
> this
> > issue?
> >
> >
> > On 10/17/06, Nathan Beyer <nb...@gmail.com> wrote:
> > >
> > > Are there any obvious bugs in Harmony that are of note? Perhaps some
> > > unexpected NPEs?
> > >
> > > On 10/16/06, Leo Li <li...@gmail.com> wrote:
> > > > Hi, all:
> > > > I have ran the tests of Derby on Harmony, both on windows xp
> and
> > > > ubuntu. The result is similar.
> > > >
> > > > On windows,
> > > > 579 Tests Run
> > > > 73% Pass (428 tests passed)
> > > > 27% Fail (151 tests failed)
> > > > 9 Suites skipped
> > > > On linux,
> > > > 579 Tests Run
> > > > 74% Pass (430 tests passed)
> > > > 26% Fail (149 tests failed)
> > > > 9 Suites skipped
> > > >
> > > > The result seems a little frustrating since Sian had 82% passed
> > > several
> > > > months before. The version of derby I ran is at revision 464429. The
> > > lost
> > > > passing rate may be due to the modified code and added testcases.
> > > >
> > > > To Sian: I made a new patch in the attachment which is an
> updated
> > > > version of the patch Sian provides. I will be appreciate if Sian
> helps
> > > to
> > > > check its validity and thus I will be assured that the lost passing
> > rate
> > > is
> > > > irrelevant to our modification.
> > > >
> > > > When I can confirm everything is all right, I will then put the
> > > ongoing
> > > > progress result on Harmony's wiki page.
> > > >
> > > > Thank you.
> > > >
> > > > --
> > > > Leo Li
> > > > China Software Development Lab, IBM
> > > >
> ---------------------------------------------------------------------
> > > > Terms of use :
> > > > http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail:
> > > > harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail:
> > > > harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> >
> >
>
>
> --
> Mikhail Fursov
>
>
--
Leo Li
China Software Development Lab, IBM
Re: [classlib]Harmony passes 73% on Derby.
Posted by Mikhail Fursov <mi...@gmail.com>.
Leo, just a simple interest, could you try to run harmony with -Xem:opt,
-Xem:jet, -Xem:server, -Xem:server_static options? These are all modes drlvm
supports today except the default one (-Xem:client)
For example if the test passes with Jitrino.JET (-Xem:jet) and fails with
Jitrino.OPT (-Xem:opt) this is a good candidate for JIT bug
On 10/17/06, Leo Li <li...@gmail.com> wrote:
>
> Hi,
> The result of Derby is not so obvious, since its test is not junit but
> some a serial of stand-alone java program calling each other. And the
> comparing result is a pre-stored data file. So I have to use sometime to
> set
> aside all the irrelevant code to spot those related with Harmony classlib.
> What makes things worse is that as a product, all the exception has
> been
> caught...
> Is there someone who has good idea to pick up problem quickly in this
> issue?
>
>
> On 10/17/06, Nathan Beyer <nb...@gmail.com> wrote:
> >
> > Are there any obvious bugs in Harmony that are of note? Perhaps some
> > unexpected NPEs?
> >
> > On 10/16/06, Leo Li <li...@gmail.com> wrote:
> > > Hi, all:
> > > I have ran the tests of Derby on Harmony, both on windows xp and
> > > ubuntu. The result is similar.
> > >
> > > On windows,
> > > 579 Tests Run
> > > 73% Pass (428 tests passed)
> > > 27% Fail (151 tests failed)
> > > 9 Suites skipped
> > > On linux,
> > > 579 Tests Run
> > > 74% Pass (430 tests passed)
> > > 26% Fail (149 tests failed)
> > > 9 Suites skipped
> > >
> > > The result seems a little frustrating since Sian had 82% passed
> > several
> > > months before. The version of derby I ran is at revision 464429. The
> > lost
> > > passing rate may be due to the modified code and added testcases.
> > >
> > > To Sian: I made a new patch in the attachment which is an updated
> > > version of the patch Sian provides. I will be appreciate if Sian helps
> > to
> > > check its validity and thus I will be assured that the lost passing
> rate
> > is
> > > irrelevant to our modification.
> > >
> > > When I can confirm everything is all right, I will then put the
> > ongoing
> > > progress result on Harmony's wiki page.
> > >
> > > Thank you.
> > >
> > > --
> > > Leo Li
> > > China Software Development Lab, IBM
> > > ---------------------------------------------------------------------
> > > Terms of use :
> > > http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail:
> > > harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail:
> > > harmony-dev-help@incubator.apache.org
> > >
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>
--
Mikhail Fursov
Re: [classlib]Harmony passes 73% on Derby.
Posted by Leo Li <li...@gmail.com>.
Hi,
The result of Derby is not so obvious, since its test is not junit but
some a serial of stand-alone java program calling each other. And the
comparing result is a pre-stored data file. So I have to use sometime to set
aside all the irrelevant code to spot those related with Harmony classlib.
What makes things worse is that as a product, all the exception has been
caught...
Is there someone who has good idea to pick up problem quickly in this
issue?
On 10/17/06, Nathan Beyer <nb...@gmail.com> wrote:
>
> Are there any obvious bugs in Harmony that are of note? Perhaps some
> unexpected NPEs?
>
> On 10/16/06, Leo Li <li...@gmail.com> wrote:
> > Hi, all:
> > I have ran the tests of Derby on Harmony, both on windows xp and
> > ubuntu. The result is similar.
> >
> > On windows,
> > 579 Tests Run
> > 73% Pass (428 tests passed)
> > 27% Fail (151 tests failed)
> > 9 Suites skipped
> > On linux,
> > 579 Tests Run
> > 74% Pass (430 tests passed)
> > 26% Fail (149 tests failed)
> > 9 Suites skipped
> >
> > The result seems a little frustrating since Sian had 82% passed
> several
> > months before. The version of derby I ran is at revision 464429. The
> lost
> > passing rate may be due to the modified code and added testcases.
> >
> > To Sian: I made a new patch in the attachment which is an updated
> > version of the patch Sian provides. I will be appreciate if Sian helps
> to
> > check its validity and thus I will be assured that the lost passing rate
> is
> > irrelevant to our modification.
> >
> > When I can confirm everything is all right, I will then put the
> ongoing
> > progress result on Harmony's wiki page.
> >
> > Thank you.
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> > ---------------------------------------------------------------------
> > Terms of use :
> > http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail:
> > harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail:
> > harmony-dev-help@incubator.apache.org
> >
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Leo Li
China Software Development Lab, IBM
Re: [classlib]Harmony passes 73% on Derby.
Posted by Nathan Beyer <nb...@gmail.com>.
Are there any obvious bugs in Harmony that are of note? Perhaps some
unexpected NPEs?
On 10/16/06, Leo Li <li...@gmail.com> wrote:
> Hi, all:
> I have ran the tests of Derby on Harmony, both on windows xp and
> ubuntu. The result is similar.
>
> On windows,
> 579 Tests Run
> 73% Pass (428 tests passed)
> 27% Fail (151 tests failed)
> 9 Suites skipped
> On linux,
> 579 Tests Run
> 74% Pass (430 tests passed)
> 26% Fail (149 tests failed)
> 9 Suites skipped
>
> The result seems a little frustrating since Sian had 82% passed several
> months before. The version of derby I ran is at revision 464429. The lost
> passing rate may be due to the modified code and added testcases.
>
> To Sian: I made a new patch in the attachment which is an updated
> version of the patch Sian provides. I will be appreciate if Sian helps to
> check its validity and thus I will be assured that the lost passing rate is
> irrelevant to our modification.
>
> When I can confirm everything is all right, I will then put the ongoing
> progress result on Harmony's wiki page.
>
> Thank you.
>
> --
> Leo Li
> China Software Development Lab, IBM
> ---------------------------------------------------------------------
> Terms of use :
> http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
>
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib]Harmony passes 73% on Derby.
Posted by Leo Li <li...@gmail.com>.
Hi, all:
One thing to correct:
Occasionally, I found that the decreased rate of pass is mainly due to
the wrongly stastics that derby team uses in tests.
The number of failed tests are calculated by the lines of a fail file,
whereas several failed tests with error message uses two lines !!!
After some tedious manual work, the passing rate is actually about 80%,
with 110 fails and 428 passses.:)
On 10/17/06, Sian January <si...@googlemail.com> wrote:
>
> Hi Leo,
>
> I have had a look at your patch and it looks fine. I have also run the
> tests once or twice since April when I first did it and I saw a lower pass
> rate too, so I don't think you've done anything wrong. I did run fewer
> tests in April as I think I was using a 1.4 target, but even so there are
> still fewer passes in total (437 in April vs. 428 now) so I think there
> must
> have been some changes in Harmony to cause this.
>
> Regards,
>
> Sian
>
>
> On 17/10/06, Leo Li <li...@gmail.com> wrote:
> >
> > Hi, all:
> > I have ran the tests of Derby on Harmony, both on windows xp and
> > ubuntu. The result is similar.
> >
> > On windows,
> > 579 Tests Run
> > 73% Pass (428 tests passed)
> > 27% Fail (151 tests failed)
> > 9 Suites skipped
> > On linux,
> > 579 Tests Run
> > 74% Pass (430 tests passed)
> > 26% Fail (149 tests failed)
> > 9 Suites skipped
> >
> > The result seems a little frustrating since Sian had 82% passed
> > several months before. The version of derby I ran is at revision 464429.
> > The lost passing rate may be due to the modified code and added
> testcases.
> >
> > To Sian: I made a new patch in the attachment which is an updated
> > version of the patch Sian provides. I will be appreciate if Sian helps
> to
> > check its validity and thus I will be assured that the lost passing rate
> is
> > irrelevant to our modification.
> >
> > When I can confirm everything is all right, I will then put the
> > ongoing progress result on Harmony's wiki page.
> >
> > Thank you.
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
> >
>
>
> --
> Sian January
>
> IBM Java Technology Centre, UK
>
>
--
Leo Li
China Software Development Lab, IBM
Re: [classlib]Harmony passes 73% on Derby.
Posted by Sian January <si...@googlemail.com>.
Hi Leo,
I have had a look at your patch and it looks fine. I have also run the
tests once or twice since April when I first did it and I saw a lower pass
rate too, so I don't think you've done anything wrong. I did run fewer
tests in April as I think I was using a 1.4 target, but even so there are
still fewer passes in total (437 in April vs. 428 now) so I think there must
have been some changes in Harmony to cause this.
Regards,
Sian
On 17/10/06, Leo Li <li...@gmail.com> wrote:
>
> Hi, all:
> I have ran the tests of Derby on Harmony, both on windows xp and
> ubuntu. The result is similar.
>
> On windows,
> 579 Tests Run
> 73% Pass (428 tests passed)
> 27% Fail (151 tests failed)
> 9 Suites skipped
> On linux,
> 579 Tests Run
> 74% Pass (430 tests passed)
> 26% Fail (149 tests failed)
> 9 Suites skipped
>
> The result seems a little frustrating since Sian had 82% passed
> several months before. The version of derby I ran is at revision 464429.
> The lost passing rate may be due to the modified code and added testcases.
>
> To Sian: I made a new patch in the attachment which is an updated
> version of the patch Sian provides. I will be appreciate if Sian helps to
> check its validity and thus I will be assured that the lost passing rate is
> irrelevant to our modification.
>
> When I can confirm everything is all right, I will then put the
> ongoing progress result on Harmony's wiki page.
>
> Thank you.
>
> --
> Leo Li
> China Software Development Lab, IBM
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
>
--
Sian January
IBM Java Technology Centre, UK