You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Tim Ellison <t....@gmail.com> on 2007/06/27 18:40:12 UTC
[testing] Trap in EUTs (was: Re: [general][M2] Code frozen)
Stepan Mishura wrote:
> On 6/27/07, Nina Rinskaya wrote:
>> Hi all,
>>
>> It looks that r549864 commit has introduced a regression:
>>
>
> I think it's too risky to rollback the commit. It looks like
> (according to the evaluation) that the impact of the regression is
> low. So I wouldn't fix it in M2. I think we should mention the issue
> in milestone's release notes and fix it after M2. Objections?
Do we know how to fix it? Weldon? It may be worth considering the
impact of the fix.
Regards,
Tim
> org.eclipse.team.tests.ccvs.core.subscriber.testConcurrentAccessToSyncSettestcase
>
>> from Eclipse Unit Tests ('teamcvs' suite) intermittently crashes on
>> Windows 2003 (x86) with the following output:
>>
>> [java] Windows reported exception: 0xc0000005
>> [java] Registers:
>> [java] EAX: 0x00410048, EBX: 0x140e9870, ECX: 0x000022a2, EDX:
>> 0x00383f30
>> [java] ESI: 0x16e3ed0c, EDI: 0x0b3943f0, ESP: 0x10d6fa78, EBP:
>> 0x13f461f4
>> [java] EIP: 0x1000430c
>> [java] Stack trace:
>> [java] 0: hythread_thin_monitor_exit (??:-1)
>> [java] <end of stack trace>
>>
>> The testcase deals with synchronized resources, and probably the crash
>> happens whyle adding ~1000-5000th element to the synchronized collection.
>>
>> The issue impacts only 1 testcase, but the whole suite (170 tests) is
>> reported as crashed, it reduces EUT pass rate by 0,4%.
>>
>> The issue was not found earlier because it is an intermittent issue
>> (but it
>> is often reproducible, ~3 times per 5 runs).
>>
>> Thanks.
>>
>> --
>> Nina
>>
>> On 6/24/07, Mikhail Loenko <ml...@gmail.com> wrote:
>> >
>> > We have passed our code freeze date for M2
>> >
>> > Last time we executed Tim's policy that each commit should have
>> > approval from two committers, let's follow the same way -- no more
>> > commits please, without agreement from two committers on the dev list.
>> >
>> > Now let's test what we have and see if any critical bugs were
>> > introduced just before the code freeze. Please raise anything you
>> > think is critical/blocker for consideration.
>> >
>> > Thanks,
>> > Mikhail
>
Re: [testing] Trap in EUTs (was: Re: [general][M2] Code frozen)
Posted by Nina Rinskaya <ni...@gmail.com>.
Hi all,
I've just created HARMONY-4298 to track this issue.
Thanks.
--
Nina
On 6/28/07, Nina Rinskaya <ni...@gmail.com> wrote:
>
> Weldon and all,
>
> It looks that I forgot to give the link to the Wiki page that describes
> how to run Eclipse Unit Tests (all together and separately):
> http://wiki.apache.org/harmony/Eclipse_Unit_Tests_Pass_on_DRLVM#How_to_run_EUT.
>
>
> The test that fails is
> org.eclipse.team.tests.ccvs.core.subscriber.testConcurrentAccessToSyncSet,
> it belongs to the 'teamcvs' suite. Please note that this suite requires a
> test CVS repository. If you use Build&Test infrastructure scrips to run the
> suite, you will need to specify in the framework.local.properties file:
>
> eut.parameters.optional.testsr=teamcvs
> eut.parameters.optional.cvs_user=<the account used to connect to the
> cvs repository>
> eut.parameters.optional.cvs_password= <the account password>
> eut.parameters.optional.cvs_host=<the name of the cvs server>
> eut.parameters.optional.cvs_root=<the repository path>
>
> If you follow instructions on how to run a single test at
> http://wiki.apache.org/harmony/Eclipse_Unit_Tests_Pass_on_DRLVM#How_to_run_EUTyou will need the following additional steps to do that are specific for
> this suite:
>
> When you perform step 4 ("Running the Test Suite from Eclipse SDK"):
> 1. select "org.eclipse.team.tests.cvs.core" as test plug-in to import; if
> Eclipse reports compilation errors, please add
> 'eclipse-testing/eclipse/plugins/org.eclipse.core.filesystem*.jar' to the
> project's build path;
> 2. modify file org.eclipse.team.tests.ccvs.core.CVSTestSetup.java: replace
> REPOSITORY_LOCATION = System.getProperty("eclipse.cvs.repository"); with
> REPOSITORY_LOCATION = ":pserver:cvs_username:cvs_password@cvs_host:cvs_root";
> 3. select org.eclipse.team.tests.ccvs.core.AllTests as launching class
>
> Please let me know if you have any questions.
>
> --
> Nina
>
> On 6/28/07, Weldon Washburn <we...@gmail.com> wrote:
> >
> > On 6/27/07, Tim Ellison <t....@gmail.com> wrote:
> > >
> > > Stepan Mishura wrote:
> > > > On 6/27/07, Nina Rinskaya wrote:
> > > >> Hi all,
> > > >>
> > > >> It looks that r549864 commit has introduced a regression:
> > > >>
> > > >
> > > > I think it's too risky to rollback the commit. It looks like
> > > > (according to the evaluation) that the impact of the regression is
> > > > low. So I wouldn't fix it in M2. I think we should mention the issue
> > > > in milestone's release notes and fix it after M2. Objections?
> > >
> > > Do we know how to fix it? Weldon? It may be worth considering the
> > > impact of the fix.
> >
> >
> > I recommend we do not fix this bug in M2. This is based on looking at
> > org.eclipse.team.tests.ccvs.core.subscriber.* source code for 20
> > minutes.
> > Its probably 2+ days work just to get the unit test to run standalone
> > and
> > exhibit the failure frequently enough to start debugging. Then 2+ days
> > to
> > develop a patch. And finally a few days to run all M2 regression tests
> > to
> > verify the new patch does not introduce new bugs. This takes us way
> > past
> > the deadline for M2.
> >
> > Regards,
> > > Tim
> > >
> > >
> > > >
> > >
> > org.eclipse.team.tests.ccvs.core.subscriber.testConcurrentAccessToSyncSettestcase
> > > >
> > > >> from Eclipse Unit Tests ('teamcvs' suite) intermittently crashes on
> >
> > > >> Windows 2003 (x86) with the following output:
> > > >>
> > > >> [java] Windows reported exception: 0xc0000005
> > > >> [java] Registers:
> > > >> [java] EAX: 0x00410048, EBX: 0x140e9870, ECX: 0x000022a2,
> > EDX:
> > > >> 0x00383f30
> > > >> [java] ESI: 0x16e3ed0c, EDI: 0x0b3943f0, ESP: 0x10d6fa78,
> > EBP:
> > > >> 0x13f461f4
> > > >> [java] EIP: 0x1000430c
> > > >> [java] Stack trace:
> > > >> [java] 0: hythread_thin_monitor_exit (??:-1)
> > > >> [java] <end of stack trace>
> > > >>
> > > >> The testcase deals with synchronized resources, and probably the
> > crash
> > > >> happens whyle adding ~1000-5000th element to the synchronized
> > > collection.
> > > >>
> > > >> The issue impacts only 1 testcase, but the whole suite (170 tests)
> > is
> > > >> reported as crashed, it reduces EUT pass rate by 0,4%.
> > > >>
> > > >> The issue was not found earlier because it is an intermittent issue
> > > >> (but it
> > > >> is often reproducible, ~3 times per 5 runs).
> > > >>
> > > >> Thanks.
> > > >>
> > > >> --
> > > >> Nina
> > > >>
> > > >> On 6/24/07, Mikhail Loenko <ml...@gmail.com> wrote:
> > > >> >
> > > >> > We have passed our code freeze date for M2
> > > >> >
> > > >> > Last time we executed Tim's policy that each commit should have
> > > >> > approval from two committers, let's follow the same way -- no
> > more
> > > >> > commits please, without agreement from two committers on the dev
> > > list.
> > > >> >
> > > >> > Now let's test what we have and see if any critical bugs were
> > > >> > introduced just before the code freeze. Please raise anything you
> > > >> > think is critical/blocker for consideration.
> > > >> >
> > > >> > Thanks,
> > > >> > Mikhail
> > > >
> > >
> >
> >
> >
> > --
> > Weldon Washburn
> >
>
>
Re: [testing] Trap in EUTs (was: Re: [general][M2] Code frozen)
Posted by Nina Rinskaya <ni...@gmail.com>.
Weldon and all,
It looks that I forgot to give the link to the Wiki page that describes how
to run Eclipse Unit Tests (all together and separately):
http://wiki.apache.org/harmony/Eclipse_Unit_Tests_Pass_on_DRLVM#How_to_run_EUT.
The test that fails is
org.eclipse.team.tests.ccvs.core.subscriber.testConcurrentAccessToSyncSet,
it belongs to the 'teamcvs' suite. Please note that this suite requires a
test CVS repository. If you use Build&Test infrastructure scrips to run the
suite, you will need to specify in the framework.local.properties file:
eut.parameters.optional.testsr=teamcvs
eut.parameters.optional.cvs_user=<the account used to connect to the cvs
repository>
eut.parameters.optional.cvs_password=<the account password>
eut.parameters.optional.cvs_host=<the name of the cvs server>
eut.parameters.optional.cvs_root=<the repository path>
If you follow instructions on how to run a single test at
http://wiki.apache.org/harmony/Eclipse_Unit_Tests_Pass_on_DRLVM#How_to_run_EUTyou
will need the following additional steps to do that are specific for
this suite:
When you perform step 4 ("Running the Test Suite from Eclipse SDK"):
1. select "org.eclipse.team.tests.cvs.core" as test plug-in to import; if
Eclipse reports compilation errors, please add
'eclipse-testing/eclipse/plugins/org.eclipse.core.filesystem*.jar' to the
project's build path;
2. modify file org.eclipse.team.tests.ccvs.core.CVSTestSetup.java: replace
REPOSITORY_LOCATION = System.getProperty("eclipse.cvs.repository"); with
REPOSITORY_LOCATION = ":pserver:cvs_username:cvs_password@cvs
_host:cvs_root";
3. select org.eclipse.team.tests.ccvs.core.AllTests as launching class
Please let me know if you have any questions.
--
Nina
On 6/28/07, Weldon Washburn <we...@gmail.com> wrote:
>
> On 6/27/07, Tim Ellison <t....@gmail.com> wrote:
> >
> > Stepan Mishura wrote:
> > > On 6/27/07, Nina Rinskaya wrote:
> > >> Hi all,
> > >>
> > >> It looks that r549864 commit has introduced a regression:
> > >>
> > >
> > > I think it's too risky to rollback the commit. It looks like
> > > (according to the evaluation) that the impact of the regression is
> > > low. So I wouldn't fix it in M2. I think we should mention the issue
> > > in milestone's release notes and fix it after M2. Objections?
> >
> > Do we know how to fix it? Weldon? It may be worth considering the
> > impact of the fix.
>
>
> I recommend we do not fix this bug in M2. This is based on looking at
> org.eclipse.team.tests.ccvs.core.subscriber.* source code for 20 minutes.
> Its probably 2+ days work just to get the unit test to run standalone and
> exhibit the failure frequently enough to start debugging. Then 2+ days to
> develop a patch. And finally a few days to run all M2 regression tests to
> verify the new patch does not introduce new bugs. This takes us way past
> the deadline for M2.
>
> Regards,
> > Tim
> >
> >
> > >
> >
> org.eclipse.team.tests.ccvs.core.subscriber.testConcurrentAccessToSyncSettestcase
> > >
> > >> from Eclipse Unit Tests ('teamcvs' suite) intermittently crashes on
> > >> Windows 2003 (x86) with the following output:
> > >>
> > >> [java] Windows reported exception: 0xc0000005
> > >> [java] Registers:
> > >> [java] EAX: 0x00410048, EBX: 0x140e9870, ECX: 0x000022a2,
> EDX:
> > >> 0x00383f30
> > >> [java] ESI: 0x16e3ed0c, EDI: 0x0b3943f0, ESP: 0x10d6fa78,
> EBP:
> > >> 0x13f461f4
> > >> [java] EIP: 0x1000430c
> > >> [java] Stack trace:
> > >> [java] 0: hythread_thin_monitor_exit (??:-1)
> > >> [java] <end of stack trace>
> > >>
> > >> The testcase deals with synchronized resources, and probably the
> crash
> > >> happens whyle adding ~1000-5000th element to the synchronized
> > collection.
> > >>
> > >> The issue impacts only 1 testcase, but the whole suite (170 tests) is
> > >> reported as crashed, it reduces EUT pass rate by 0,4%.
> > >>
> > >> The issue was not found earlier because it is an intermittent issue
> > >> (but it
> > >> is often reproducible, ~3 times per 5 runs).
> > >>
> > >> Thanks.
> > >>
> > >> --
> > >> Nina
> > >>
> > >> On 6/24/07, Mikhail Loenko <ml...@gmail.com> wrote:
> > >> >
> > >> > We have passed our code freeze date for M2
> > >> >
> > >> > Last time we executed Tim's policy that each commit should have
> > >> > approval from two committers, let's follow the same way -- no more
> > >> > commits please, without agreement from two committers on the dev
> > list.
> > >> >
> > >> > Now let's test what we have and see if any critical bugs were
> > >> > introduced just before the code freeze. Please raise anything you
> > >> > think is critical/blocker for consideration.
> > >> >
> > >> > Thanks,
> > >> > Mikhail
> > >
> >
>
>
>
> --
> Weldon Washburn
>
Re: [testing] Trap in EUTs (was: Re: [general][M2] Code frozen)
Posted by Weldon Washburn <we...@gmail.com>.
On 6/27/07, Tim Ellison <t....@gmail.com> wrote:
>
> Stepan Mishura wrote:
> > On 6/27/07, Nina Rinskaya wrote:
> >> Hi all,
> >>
> >> It looks that r549864 commit has introduced a regression:
> >>
> >
> > I think it's too risky to rollback the commit. It looks like
> > (according to the evaluation) that the impact of the regression is
> > low. So I wouldn't fix it in M2. I think we should mention the issue
> > in milestone's release notes and fix it after M2. Objections?
>
> Do we know how to fix it? Weldon? It may be worth considering the
> impact of the fix.
I recommend we do not fix this bug in M2. This is based on looking at
org.eclipse.team.tests.ccvs.core.subscriber.* source code for 20 minutes.
Its probably 2+ days work just to get the unit test to run standalone and
exhibit the failure frequently enough to start debugging. Then 2+ days to
develop a patch. And finally a few days to run all M2 regression tests to
verify the new patch does not introduce new bugs. This takes us way past
the deadline for M2.
Regards,
> Tim
>
>
> >
> org.eclipse.team.tests.ccvs.core.subscriber.testConcurrentAccessToSyncSettestcase
> >
> >> from Eclipse Unit Tests ('teamcvs' suite) intermittently crashes on
> >> Windows 2003 (x86) with the following output:
> >>
> >> [java] Windows reported exception: 0xc0000005
> >> [java] Registers:
> >> [java] EAX: 0x00410048, EBX: 0x140e9870, ECX: 0x000022a2, EDX:
> >> 0x00383f30
> >> [java] ESI: 0x16e3ed0c, EDI: 0x0b3943f0, ESP: 0x10d6fa78, EBP:
> >> 0x13f461f4
> >> [java] EIP: 0x1000430c
> >> [java] Stack trace:
> >> [java] 0: hythread_thin_monitor_exit (??:-1)
> >> [java] <end of stack trace>
> >>
> >> The testcase deals with synchronized resources, and probably the crash
> >> happens whyle adding ~1000-5000th element to the synchronized
> collection.
> >>
> >> The issue impacts only 1 testcase, but the whole suite (170 tests) is
> >> reported as crashed, it reduces EUT pass rate by 0,4%.
> >>
> >> The issue was not found earlier because it is an intermittent issue
> >> (but it
> >> is often reproducible, ~3 times per 5 runs).
> >>
> >> Thanks.
> >>
> >> --
> >> Nina
> >>
> >> On 6/24/07, Mikhail Loenko <ml...@gmail.com> wrote:
> >> >
> >> > We have passed our code freeze date for M2
> >> >
> >> > Last time we executed Tim's policy that each commit should have
> >> > approval from two committers, let's follow the same way -- no more
> >> > commits please, without agreement from two committers on the dev
> list.
> >> >
> >> > Now let's test what we have and see if any critical bugs were
> >> > introduced just before the code freeze. Please raise anything you
> >> > think is critical/blocker for consideration.
> >> >
> >> > Thanks,
> >> > Mikhail
> >
>
--
Weldon Washburn
Re: [testing] Trap in EUTs (was: Re: [general][M2] Code frozen)
Posted by Weldon Washburn <we...@gmail.com>.
On 6/27/07, Tim Ellison <t....@gmail.com> wrote:
>
> Stepan Mishura wrote:
> > On 6/27/07, Nina Rinskaya wrote:
> >> Hi all,
> >>
> >> It looks that r549864 commit has introduced a regression:
> >>
> >
> > I think it's too risky to rollback the commit. It looks like
> > (according to the evaluation) that the impact of the regression is
> > low. So I wouldn't fix it in M2. I think we should mention the issue
> > in milestone's release notes and fix it after M2. Objections?
>
> Do we know how to fix it? Weldon? It may be worth considering the
> impact of the fix.
I have just now started looking at this bug. I hope to have an update
sometime today.
- Weldon
Regards,
> Tim
>
>
> >
> org.eclipse.team.tests.ccvs.core.subscriber.testConcurrentAccessToSyncSettestcase
> >
> >> from Eclipse Unit Tests ('teamcvs' suite) intermittently crashes on
> >> Windows 2003 (x86) with the following output:
> >>
> >> [java] Windows reported exception: 0xc0000005
> >> [java] Registers:
> >> [java] EAX: 0x00410048, EBX: 0x140e9870, ECX: 0x000022a2, EDX:
> >> 0x00383f30
> >> [java] ESI: 0x16e3ed0c, EDI: 0x0b3943f0, ESP: 0x10d6fa78, EBP:
> >> 0x13f461f4
> >> [java] EIP: 0x1000430c
> >> [java] Stack trace:
> >> [java] 0: hythread_thin_monitor_exit (??:-1)
> >> [java] <end of stack trace>
> >>
> >> The testcase deals with synchronized resources, and probably the crash
> >> happens whyle adding ~1000-5000th element to the synchronized
> collection.
> >>
> >> The issue impacts only 1 testcase, but the whole suite (170 tests) is
> >> reported as crashed, it reduces EUT pass rate by 0,4%.
> >>
> >> The issue was not found earlier because it is an intermittent issue
> >> (but it
> >> is often reproducible, ~3 times per 5 runs).
> >>
> >> Thanks.
> >>
> >> --
> >> Nina
> >>
> >> On 6/24/07, Mikhail Loenko <ml...@gmail.com> wrote:
> >> >
> >> > We have passed our code freeze date for M2
> >> >
> >> > Last time we executed Tim's policy that each commit should have
> >> > approval from two committers, let's follow the same way -- no more
> >> > commits please, without agreement from two committers on the dev
> list.
> >> >
> >> > Now let's test what we have and see if any critical bugs were
> >> > introduced just before the code freeze. Please raise anything you
> >> > think is critical/blocker for consideration.
> >> >
> >> > Thanks,
> >> > Mikhail
> >
>
--
Weldon Washburn