Re: [general] aiming no regression

[Vladimir Ivanov <iv...@gmail.com>]

On 1/23/07, Mikhail Loenko <ml...@gmail.com> wrote:
> 2007/1/18, Vladimir Ivanov <iv...@gmail.com>:
> > On 12/20/06, Geir Magnusson Jr. <ge...@pobox.com> wrote:
> >
> > > Mikhail Loenko wrote:
> > > > I suggest that we don't exclude more tests listed in 2438 -- it seems
> > > > like any
> > > > swiing test can fail
> > > >
> > > > Instead we may remove all swing tests from CC when run on J9 and try to
> > > > fix the
> > > > problem
> > >
> > > +1
> > >
> > > geir
> >
> >
> >  Actually, the 'swing' tests sometimes intermittently failed on the DRLVM
> > too (for example, I was able to reproduce vm crash for test discusssed at
> > Jan15 in topic '[classlib] new tests crashes':
> > <snip>
> >    [junit] free(): invalid pointer 0x9db42d8!
> >    [junit] free(): invalid pointer 0x9db72d0!
> >    [junit] SIGSEGV in VM code.
> >    [junit] Stack trace:
> >    [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
> >    [junit] Tests FAILED (timeout)
> > ...
> > /export/users/viv/trunk/cc/projects/classlib/trunk/make/build-test.xml:133:
> > There were test crashes:
> > /export/users/viv/trunk/cc/projects/classlib/trunk/build/test_report/TEST-
> > javax.swing.LayoutFocusTraversalPolicyTest.xml
> >
> >
> >
> > The question is: should we exclude swing tests from the cruise control
> > totally (while undefined problem into swing will be fixed)
>
> Yes! If it fails DRLVM let's exclude them from there either

The 'swing' module excluded now from the CC cycle on systems number 3,
4 and 5. When we will include it to regular runs?

 thanks, Vladimir

>
> Thanks,
> Mikhail
>
>
>
> or continue to
> > add tests one-by-one to the exclude list?
> >
> > Note, when these test were excluded (thanks to Mark!) no new intermittently
> > failed test were detected in the swing for 3 days.
> >
> >
> >  Thanks, Vladimir
> >
> >
> > > >
> > > > Thanks,
> > > > Mikhail
> > > >
> > > >
> > > > 2006/12/20, Vladimir Ivanov <iv...@gmail.com>:
> > > >> Actually, I was able to see these failures on swing tests only. But
> > > >> even for
> > > >> swing these failures reproduced intermittently and only when all swing
> > > >> tests
> > > >> run in the one VM.
> > > >>
> > > >>
> > > >>
> > > >>  Thanks, Vladimir
> > > >>
> > > >>
> > > >> On 12/19/06, Mikhail Loenko <ml...@gmail.com> wrote:
> > > >> >
> > > >> > have you seen this stack when other tests run? maybe gui
> > > >> > breaks something causing the failure? Are you able to reproduce the
> > > >> > problem?
> > > >> >
> > > >> > Thanks,
> > > >> > Mikhail
> > > >> >
> > > >> > 2006/12/19, Vladimir Ivanov <ivavladimir@gmail.com >:
> > > >> > > On 12/19/06, Ivanov, Alexey A < alexey.a.ivanov@intel.com> wrote:
> > > >> > >
> > > >> > > >
> > > >> > > > There's only one GUI test in your list:
> > > >> javax.swing.JToggleButtonTest.
> > > >> > > > The others test text model, and this particular tests don't use
> > > any
> > > >> > > > swing UI components at all.
> > > >> > > >
> > > >> > > > If I remember your reports correctly, the latter three tests fail
> > > >> > > > because of some serialization failure.
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > Yes, testParamString at javax.swing.JToggleButtonTest and
> > > >> > testSerializable
> > > >> > > for other tests. But actually the stack trace is similar (below) so
> > > I
> > > >> > think
> > > >> > > it not gui test problem. It is just reproduce this issue.
> > > >> > >
> > > >> > >  Thanks, Vladimir
> > > >> > > Stack trace:
> > > >> > > Test: testParamStringClass: javax.swing.JToggleButtonTest
> > > >> > > java.lang.ArrayIndexOutOfBoundsException
> > > >> > > at java.util.Arrays.mergeSort(Arrays.java:2553)
> > > >> > > at java.util.Arrays.mergeSort (Arrays.java :2516)
> > > >> > > at java.util.Arrays.sort(Arrays.java:2872)
> > > >> > > at java.util.Arrays.sort(Arrays.java:2889)
> > > >> > > at java.beans.BeanInfoWrapper.getPropertyDescriptors(
> > > >> > BeanInfoWrapper.java
> > > >> > > :77)
> > > >> > > at java.beans.BeanInfoWrapper.getPropertyDescriptors(
> > > >> > BeanInfoWrapper.java
> > > >> > > :74)
> > > >> > > at javax.swing.JComponent.paramString (JComponent.java:1334)
> > > >> > > at java.awt.Component.toString(Component.java:166)
> > > >> > > at
> > > >> javax.swing.JToggleButtonTest.testParamString(JToggleButtonTest.java
> > > >> > :64)
> > > >> > > at
> > > >> java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
> > > >> > > at
> > > >> javax.swing.BasicSwingTestCase.runBareSuper(BasicSwingTestCase.java
> > > :117)
> > > >> > > at javax.swing.SwingTestCase$1.run(SwingTestCase.java:45)
> > > >> > > at
> > > >> java.awt.event.InvocationEvent.runAndNotify(InvocationEvent.java:92)
> > > >> > > at java.awt.event.InvocationEvent.dispatch (InvocationEvent.java:81)
> > > >> > > at java.awt.EventQueueCore.dispatchEventImpl(EventQueueCore.java
> > > :133)
> > > >> > > at java.awt.EventQueue.dispatchEvent(EventQueue.java:144)
> > > >> > > at
> > > >> java.awt.EventDispatchThread.runModalLoop(EventDispatchThread.java:75)
> > > >> > > at java.awt.EventDispatchThread.run(EventDispatchThread.java:48)
> > > >> > >
> > > >> > > Test: testSerializableClass:
> > > >> > > javax.swing.text.AbstractDocument_SerializationTest
> > > >> > > java.lang.ArrayIndexOutOfBoundsException
> > > >> > > at java.util.Arrays.mergeSort(Arrays.java:2553)
> > > >> > > at java.util.Arrays.mergeSort(Arrays.java:2516)
> > > >> > > at java.util.Arrays.mergeSort(Arrays.java:2517)
> > > >> > > at java.util.Arrays.sort(Arrays.java:2872)
> > > >> > > at java.util.Arrays.sort (Arrays.java:2889)
> > > >> > > at java.io.ObjectStreamClass.computeSerialVersionUID(
> > > >> > ObjectStreamClass.java
> > > >> > > :54)
> > > >> > > at java.io.ObjectStreamClass.addToCache (ObjectStreamClass.java
> > > :211)
> > > >> > > at java.io.ObjectStreamClass.lookupStreamClass(
> > > ObjectStreamClass.java
> > > >> > :937)
> > > >> > > at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java :90)
> > > >> > > at java.io.ObjectStreamClass.addToCache(ObjectStreamClass.java :23)
> > > >> > > at java.io.ObjectStreamClass.lookupStreamClass(
> > > ObjectStreamClass.java
> > > >> > :937)
> > > >> > > at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:90)
> > > >> > > at java.io.ObjectOutputStream.writeClassDescForClass(
> > > >> > ObjectOutputStream.java
> > > >> > > :110)
> > > >> > > at java.io.ObjectOutputStream.writeNewObject(
> > > ObjectOutputStream.java
> > > >> > :1644)
> > > >> > > at java.io.ObjectOutputStream.writeObjectInternal(
> > > >> > ObjectOutputStream.java
> > > >> > > :1956)
> > > >> > > at java.io.ObjectOutputStream.writeObject
> > > >> (ObjectOutputStream.java:1785)
> > > >> > > at
> > > >> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:1749)
> > > >> > > at javax.swing.BasicSwingTestCase.serializeObject(
> > > >> > BasicSwingTestCase.java
> > > >> > > :496)
> > > >> > > at javax.swing.SerializableTestCase.setUp
> > > >> (SerializableTestCase.java:50)
> > > >> > > at javax.swing.text.AbstractDocument_SerializationTest.setUp
> > > >> > > (AbstractDocument_SerializationTest.java:43)
> > > >> > >
> > > >> > >
> > > >> > > > Regards,
> > > >> > > > Alexey.
> > > >> > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >On 12/18/06, Geir Magnusson Jr. < geir@pobox.com> wrote:
> > > >> > > > >>
> > > >> > > > >>
> > > >> > > > >>
> > > >> > > > >> Mikhail Loenko wrote:
> > > >> > > > >> > 2006/12/18, Geir Magnusson Jr. < geir@pobox.com >:
> > > >> > > > >> >>
> > > >> > > > >> >>
> > > >> > > > >> >> Mikhail Loenko wrote:
> > > >> > > > >> >> > 2006/12/1, Geir Magnusson Jr. < geir@pobox.com>:
> > > >> > > > >> >> >>
> > > >> > > > >> >> >>
> > > >> > > > >> >> >> Mikhail Loenko wrote:
> > > >> > > > >> >> >> > 4) We have cruise controls running classlibrary tests
> > > on
> > > >> > > > DRLVM.
> > > >> > > > >We
> > > >> > > > >> >> >> > need to decide what will we do when DRLVM+Classlib
> > > >> cruise
> > > >> > > > control
> > > >> > > > >> >> >> > reports failure.
> > > >> > > > >> >> >>
> > > >> > > > >> >> >> Stop and fix the problem.  Is there really a question
> > > >> > here?  I
> > > >> > > > >agree
> > > >> > > > >> >> >
> > > >> > > > >> >> > Yes, there is a question here. "Stop and fix" includes
> > > >> > > > "discuss".
> > > >> > > > >But
> > > >> > > > >>
> > > >> > > > >> >> > as we now know discussion may take several days. And
> > > while
> > > >> > some
> > > >> > > > >> people
> > > >> > > > >> >> > discuss what the problem is other people can't proceed
> > > with
> > > >> > > > >> >> > development and patch
> > > >> > > > >> >> > intagration.
> > > >> > > > >> >> >
> > > >> > > > >> >> > To have better pace and better CC up-time we need
> > > something
> > > >> > else
> > > >> > > > but
> > > >> > > > >> >> not
> > > >> > > > >> >> > just "stop and fix". I suggest "revert and continue"
> > > >> > > > >> >>
> > > >> > > > >> >> What's the difference, other than debating the semantics of
> > > >> > "fix"
> > > >> > > > and
> > > >> > > > >> >> "revert"?
> > > >> > > > >> >>
> > > >> > > > >> >> We all agree - but I still don't think you're clearly
> > > stating
> > > >> > the
> > > >> > > > >> >> problem.  I think that the core problem is that we don't
> > > >> > > > immediately
> > > >> > > > >> >> react to CC failure.
> > > >> > > > >> >>
> > > >> > > > >> >> Immediately reacting to CC failure should be the first
> > > >> order of
> > > >> > > > the
> > > >> > > > >day
> > > >> > > > >> >> here.  Reacting to me is making the decision, quickly,
> > > about
> > > >> > > > either
> > > >> > > > >> >> rolling back the change ("reverting") or doing something
> > > >> else.
> > > >> > > > The
> > > >> > > > >key
> > > >> > > > >>
> > > >> > > > >> >> is being responsive.
> > > >> > > > >> >>
> > > >> > > > >> >> It seems that what happens is that we wait, and then sets
> > > of
> > > >> > > > changes
> > > >> > > > >> >> pile up, and I think that doing mass rollbacks at that
> > > point
> > > >> > will
> > > >> > > > >solve
> > > >> > > > >> >> it, but make a mess.
> > > >> > > > >> >>
> > > >> > > > >> >> The example of what I envision is when I broke the build in
> > >
> > > >> > DRLVM,
> > > >> > > > >> >> Gregory told me immediately, and I fixed immediately - w/o
> > > a
> > > >> > > > rollback.
> > > >> > > > >> >>
> > > >> > > > >> >>
> > > >> > > > >> >> All I'm saying is :
> > > >> > > > >> >>
> > > >> > > > >> >> 1) We need to be far better with reaction time
> > > >> > > > >> >
> > > >> > > > >> > I would say we need to be far better with fixing/reverting
> > > >> time.
> > > >> > > > >> > If we reacted immediately and than discussed for two weeks
> > > >> -- we
> > > >> > > > would
> > > >> > > > >> not
> > > >> > > > >> > be better than where we are now
> > > >> > > > >>
> > > >> > > > >> Yes, fixing/reverting is included. It's what I meant.
> > > >> > > > >>
> > > >> > > > >> >
> > > >> > > > >> >>
> > > >> > > > >> >> 2) We have intelligent people - we can be agile in this by
> > > >> > making
> > > >> > > > >> >> decisions (quickly!) on a case by case basis what to do.
> > > >> > > > >> >>
> > > >> > > > >> >> I'll also suggest that we ask each committer to check the
> > > CC
> > > >> > event
> > > >> > > > >> >> stream before committing, so you don't commit into a bad
> > > >> state
> > > >> > of
> > > >> > > > >> things.
> > > >> > > > >> >>
> > > >> > > > >> >> One of my problems is that I don't trust the CC stream, and
> > > >> > don't
> > > >> > > > >> >> clearly see it because it's mixed in the other drek of the
> > > >> > > > commits@
> > > >> > > > >> list.
> > > >> > > > >> >
> > > >> > > > >> > The problem is intermittent failures. I suggest that we
> > > >> exclude
> > > >> > > > >graphics
> > > >> > > > >> > tests
> > > >> > > > >> > from CCs and probably have CC-specific exclude lists for
> > > >> > networking
> > > >> > > > >> tests
> > > >> > > > >> > (or fix all the known intermittent failures right now :)
> > > >> > > > >>
> > > >> > > > >> good idea - works for me.
> > > >> > > > >>
> > > >> > > > >> We need to drive into stability - we've made amazing progress
> > > in
> > > >> > the
> > > >> > > > >> last two months, and now we're down to the really, really hard
> > >
> > > >> > stuff.
> > > >> > > > I
> > > >> > > > >> think that excluding them to get rock-solid CC reporting is
> > > >> step 0,
> > > >> > > > >> and then step 1 is try and grind out the intermittent
> > > failures.
> > > >> > > > >>
> > > >> > > > >> geir
> > > >> > > > >>
> > > >> > > > >>
> > > >> > > >
> > > >> > > > --
> > > >> > > > Alexey A. Ivanov
> > > >> > > > Intel Enterprise Solutions Software Division
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > > >>
> > >
> >
> >
>