You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flex.apache.org by Erik de Bruin <er...@ixsoftware.nl> on 2014/05/22 08:48:25 UTC
[Mustella] still failing, must fix
Hi,
With the commit rate to the SDK picking up, it won't do to ignore the
failing tests. Several commits seem to have broken something, and it's
getting harder to tell which caused what...
I'm calling on all recent committers to look at the changes they submitted
and please fix what's causing the broken tests!
Also, as a remark to knowingly submitting code that will cause Mustella to
fail, please don't. Create a branch, run the tests on that and discuss any
fixes to the tests in relation to that branch. We want to keep 'develop' as
stable as we can, specifically to avoid situations like the one we're in
now.
Thanks,
EdB
--
Ix Multimedia Software
Jan Luykenstraat 27
3521 VB Utrecht
T. 06-51952295
I. www.ixsoftware.nl
Re: [Mustella] still failing, must fix
Posted by Nicholas Kwiatkowski <ni...@spoon.as>.
About half of my bitmap compares fail. They always have on this machine.
It's not the fonts, it looks to be something the anti-aliasing, I believe.
This happens on runtimes > 11.3.
I'll round up some failures for you this evening.
-Nick
On Sat, May 24, 2014 at 11:59 PM, Alex Harui <ah...@adobe.com> wrote:
> Hi Nick,
>
> Does "local image" mean mx:Image, spark:Image or any bitmap compare? Are
> you sure you have the embedded font libraries set up?
>
> Can you send me a .png, bad.png and the SWF that produced the failure?
>
> Thanks,
> -Alex
>
> On 5/24/14 7:49 PM, "Nicholas Kwiatkowski" <ni...@spoon.as> wrote:
>
> >My issue is that about half of my local image tests fail, so its difficult
> >for me to get a baseline... If I have local test that fails before I make
> >changes, I skip those tests and go on...
> >
> >Instead of having another branch that we need to mess with that will be
> >polluted with failures, it might be nice to somehow instruct one of the
> >test servers to build off another branch that we are ready to have tested.
> > If that passes, then we can merge it into develop. That would encourage
> >people to use branches more often, and may clean up the testing process...
> >
> >-Nick
> >
> >
> >On Thu, May 22, 2014 at 11:46 PM, Alex Harui <ah...@adobe.com> wrote:
> >
> >>
> >>
> >> On 5/22/14 6:34 PM, "Nicholas Kwiatkowski" <ni...@spoon.as> wrote:
> >>
> >> >Whats the easiest way to request a branch to be tested in Mustella? I
> >> >have
> >> >a few branches that I've not been able to fully test in my local
> >>Mustella
> >> >(the changes include tests that have always failed in my local
> >>instance.
> >> That's a good question. However, if you've made a check in and there
> >>are
> >> new failures in the run that includes your changes, it would be a good
> >> idea to run some of those tests yourself with and without your changes
> >>and
> >> if your changes make a difference, consider reverting.
> >>
> >> My email-driven patch testing server was a bit too flaky so I've taken
> >>it
> >> down, but now that I've got an Azure machine, I'm considering conjuring
> >>up
> >> a web-app driven patch testing server if I ever find time or other
> >> volunteers can pitch in. At Adobe we used a web-app patch testing
> >>server.
> >> You visit some URL and submit your patch and you'll get an email back
> >> with the set of failures.
> >>
> >> We could also just set up a branch that folks submit to first and have
> >> jenkins run mustella on that branch, but it still gets messy when
> >>several
> >> changes are in a run.
> >>
> >> If anybody wants to help write the web-app for the patch testing server,
> >> let me know.
> >>
> >> -Alex
> >>
> >>
>
>
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
Hi Nick,
Does "local image" mean mx:Image, spark:Image or any bitmap compare? Are
you sure you have the embedded font libraries set up?
Can you send me a .png, bad.png and the SWF that produced the failure?
Thanks,
-Alex
On 5/24/14 7:49 PM, "Nicholas Kwiatkowski" <ni...@spoon.as> wrote:
>My issue is that about half of my local image tests fail, so its difficult
>for me to get a baseline... If I have local test that fails before I make
>changes, I skip those tests and go on...
>
>Instead of having another branch that we need to mess with that will be
>polluted with failures, it might be nice to somehow instruct one of the
>test servers to build off another branch that we are ready to have tested.
> If that passes, then we can merge it into develop. That would encourage
>people to use branches more often, and may clean up the testing process...
>
>-Nick
>
>
>On Thu, May 22, 2014 at 11:46 PM, Alex Harui <ah...@adobe.com> wrote:
>
>>
>>
>> On 5/22/14 6:34 PM, "Nicholas Kwiatkowski" <ni...@spoon.as> wrote:
>>
>> >Whats the easiest way to request a branch to be tested in Mustella? I
>> >have
>> >a few branches that I've not been able to fully test in my local
>>Mustella
>> >(the changes include tests that have always failed in my local
>>instance.
>> That's a good question. However, if you've made a check in and there
>>are
>> new failures in the run that includes your changes, it would be a good
>> idea to run some of those tests yourself with and without your changes
>>and
>> if your changes make a difference, consider reverting.
>>
>> My email-driven patch testing server was a bit too flaky so I've taken
>>it
>> down, but now that I've got an Azure machine, I'm considering conjuring
>>up
>> a web-app driven patch testing server if I ever find time or other
>> volunteers can pitch in. At Adobe we used a web-app patch testing
>>server.
>> You visit some URL and submit your patch and you'll get an email back
>> with the set of failures.
>>
>> We could also just set up a branch that folks submit to first and have
>> jenkins run mustella on that branch, but it still gets messy when
>>several
>> changes are in a run.
>>
>> If anybody wants to help write the web-app for the patch testing server,
>> let me know.
>>
>> -Alex
>>
>>
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
HI,
> Why is that? Can't you have a test server checkout another branch from
> time to time?
Given it takes 8 hours to you the tests a single server may be about to 3 or 4 branches in 24 hours at most, you may need to run several times to get it right lets assume 2 -3 times so that's just over 1 branch/machine/day.
Justin
Re: [Mustella] still failing, must fix
Posted by Nicholas Kwiatkowski <ni...@spoon.as>.
Why is that? Can't you have a test server checkout another branch from
time to time? As long as it was branched from develop, it should only
include changes that person was working on...
-Nick
On Sat, May 24, 2014 at 10:52 PM, Justin Mclean <ju...@classsoftware.com>wrote:
> Hi,
>
> > it might be nice to somehow instruct one of the
> > test servers to build off another branch that we are ready to have
> tested.
> > If that passes, then we can merge it into develop.
>
> Doesn't seem very scalable to me as you would need one test server per
> branch.
>
> Justin
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> it might be nice to somehow instruct one of the
> test servers to build off another branch that we are ready to have tested.
> If that passes, then we can merge it into develop.
Doesn't seem very scalable to me as you would need one test server per branch.
Justin
Re: [Mustella] still failing, must fix
Posted by Nicholas Kwiatkowski <ni...@spoon.as>.
My issue is that about half of my local image tests fail, so its difficult
for me to get a baseline... If I have local test that fails before I make
changes, I skip those tests and go on...
Instead of having another branch that we need to mess with that will be
polluted with failures, it might be nice to somehow instruct one of the
test servers to build off another branch that we are ready to have tested.
If that passes, then we can merge it into develop. That would encourage
people to use branches more often, and may clean up the testing process...
-Nick
On Thu, May 22, 2014 at 11:46 PM, Alex Harui <ah...@adobe.com> wrote:
>
>
> On 5/22/14 6:34 PM, "Nicholas Kwiatkowski" <ni...@spoon.as> wrote:
>
> >Whats the easiest way to request a branch to be tested in Mustella? I
> >have
> >a few branches that I've not been able to fully test in my local Mustella
> >(the changes include tests that have always failed in my local instance.
> That's a good question. However, if you've made a check in and there are
> new failures in the run that includes your changes, it would be a good
> idea to run some of those tests yourself with and without your changes and
> if your changes make a difference, consider reverting.
>
> My email-driven patch testing server was a bit too flaky so I've taken it
> down, but now that I've got an Azure machine, I'm considering conjuring up
> a web-app driven patch testing server if I ever find time or other
> volunteers can pitch in. At Adobe we used a web-app patch testing server.
> You visit some URL and submit your patch and you'll get an email back
> with the set of failures.
>
> We could also just set up a branch that folks submit to first and have
> jenkins run mustella on that branch, but it still gets messy when several
> changes are in a run.
>
> If anybody wants to help write the web-app for the patch testing server,
> let me know.
>
> -Alex
>
>
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
On 5/22/14 6:34 PM, "Nicholas Kwiatkowski" <ni...@spoon.as> wrote:
>Whats the easiest way to request a branch to be tested in Mustella? I
>have
>a few branches that I've not been able to fully test in my local Mustella
>(the changes include tests that have always failed in my local instance.
That's a good question. However, if you've made a check in and there are
new failures in the run that includes your changes, it would be a good
idea to run some of those tests yourself with and without your changes and
if your changes make a difference, consider reverting.
My email-driven patch testing server was a bit too flaky so I've taken it
down, but now that I've got an Azure machine, I'm considering conjuring up
a web-app driven patch testing server if I ever find time or other
volunteers can pitch in. At Adobe we used a web-app patch testing server.
You visit some URL and submit your patch and you'll get an email back
with the set of failures.
We could also just set up a branch that folks submit to first and have
jenkins run mustella on that branch, but it still gets messy when several
changes are in a run.
If anybody wants to help write the web-app for the patch testing server,
let me know.
-Alex
Re: [Mustella] still failing, must fix
Posted by Nicholas Kwiatkowski <ni...@spoon.as>.
Whats the easiest way to request a branch to be tested in Mustella? I have
a few branches that I've not been able to fully test in my local Mustella
(the changes include tests that have always failed in my local instance.
Thanks!
On Thu, May 22, 2014 at 2:48 AM, Erik de Bruin <er...@ixsoftware.nl> wrote:
> Hi,
>
> With the commit rate to the SDK picking up, it won't do to ignore the
> failing tests. Several commits seem to have broken something, and it's
> getting harder to tell which caused what...
>
> I'm calling on all recent committers to look at the changes they submitted
> and please fix what's causing the broken tests!
>
> Also, as a remark to knowingly submitting code that will cause Mustella to
> fail, please don't. Create a branch, run the tests on that and discuss any
> fixes to the tests in relation to that branch. We want to keep 'develop' as
> stable as we can, specifically to avoid situations like the one we're in
> now.
>
> Thanks,
>
> EdB
>
>
>
> --
> Ix Multimedia Software
>
> Jan Luykenstraat 27
> 3521 VB Utrecht
>
> T. 06-51952295
> I. www.ixsoftware.nl
>
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
HI,
> Ok, one week later, there are 43 (!) tests failing:
I'll take a look at the the two Alert tests tat are still failing.
Justin
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
I fixed the Alert test that were failing - what was odd was sometime the test passed and sometimes they failed due to sub pixel differences. New baseline images should fix that.
Justin
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@me.com>.
Hi,
> Are you sure you pulled and built Nick's XMLListCollection changes?
100% surre. On latest develop branch and have done a clean compile of everything in en_US and ja_JP. Git log shows his change and merge. (053f8619cd5647b867f04a19d9afe797eaccd58f and 89853fa181c6bd5f3b781cda88a4c9b967000619)
Justin
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
Are you sure you pulled and built Nick's XMLListCollection changes?
On 5/27/14 5:23 AM, "Justin Mclean" <ju...@classsoftware.com> wrote:
>Hi,
>
>Also these all pass first run for me:
>gumbo/components/MXItemRenderer
>gumbo/core/DataGroup
>
>Justin
>
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
Also these all pass first run for me:
gumbo/components/MXItemRenderer
gumbo/core/DataGroup
Justin
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
I can reproduce the Japanese test failures - but no sure what the issue is there.
Justin
Re: [Mustella] still failing, must fix
Posted by OmPrakash Muppirala <bi...@gmail.com>.
On Jun 4, 2014 9:37 AM, "Erik de Bruin" <er...@ixsoftware.nl> wrote:
>
> We're going on 3 weeks with a failing Mustella now, and lots of commits
> still being made to the repo...
>
> Are you ready to revert whatever is causing these failures? You know who
> you are...
>
+1 to revert code that is breaking Mustella. Let's move them to a branch
and make all tests pass on develop branch, please.
Thanks,
Om
> EdB
>
>
>
>
> On Mon, Jun 2, 2014 at 8:38 PM, Alex Harui <ah...@adobe.com> wrote:
>
> >
> >
> > On 6/2/14 11:25 AM, "Michael A. Labriola" <la...@digitalprimates.net>
> > wrote:
> >
> > >>Makes sense and we probably should have done that in the first place.
> > >>But since we didn't, do we change behavior and risk breaking folks or
> > >>add a flag and keep both code paths?
> > >
> > >The problem I have with two code paths is how do we choose between
them?
> > >Are we going to do a version number check or make someone compile
> > >differently, etc.? Also, FWIW as a data point, other than a test which
> > >was expecting a specific error to be thrown, I am going to highly doubt
> > >anyone would even know a change was made unless they were working
around
> > >it before. Willing to go either direction but I am always hesitant to
be
> > >dragged down by complete backward compatibility, especially for low
risk
> > >items.
> > Well, if you want to take the risk, go with the single code path and
> > comment out the mustella tests and if it breaks someone I will point
them
> > to you. I won't veto such a change.
> >
> > If it were me, I'd add the flag, default to new behavior, and set the
flag
> > in the mustella tests.
> >
> > I agree we don't need to be completely backward compatible for past
> > incorrect behavior, but I'm often reminded of how we think we won't
break
> > anybody and then do.
> >
> > -Alex
> >
> >
>
>
> --
> Ix Multimedia Software
>
> Jan Luykenstraat 27
> 3521 VB Utrecht
>
> T. 06-51952295
> I. www.ixsoftware.nl
Re: [Mustella] still failing, must fix
Posted by Erik de Bruin <er...@ixsoftware.nl>.
We're going on 3 weeks with a failing Mustella now, and lots of commits
still being made to the repo...
Are you ready to revert whatever is causing these failures? You know who
you are...
EdB
On Mon, Jun 2, 2014 at 8:38 PM, Alex Harui <ah...@adobe.com> wrote:
>
>
> On 6/2/14 11:25 AM, "Michael A. Labriola" <la...@digitalprimates.net>
> wrote:
>
> >>Makes sense and we probably should have done that in the first place.
> >>But since we didn't, do we change behavior and risk breaking folks or
> >>add a flag and keep both code paths?
> >
> >The problem I have with two code paths is how do we choose between them?
> >Are we going to do a version number check or make someone compile
> >differently, etc.? Also, FWIW as a data point, other than a test which
> >was expecting a specific error to be thrown, I am going to highly doubt
> >anyone would even know a change was made unless they were working around
> >it before. Willing to go either direction but I am always hesitant to be
> >dragged down by complete backward compatibility, especially for low risk
> >items.
> Well, if you want to take the risk, go with the single code path and
> comment out the mustella tests and if it breaks someone I will point them
> to you. I won't veto such a change.
>
> If it were me, I'd add the flag, default to new behavior, and set the flag
> in the mustella tests.
>
> I agree we don't need to be completely backward compatible for past
> incorrect behavior, but I'm often reminded of how we think we won't break
> anybody and then do.
>
> -Alex
>
>
--
Ix Multimedia Software
Jan Luykenstraat 27
3521 VB Utrecht
T. 06-51952295
I. www.ixsoftware.nl
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
On 6/2/14 11:25 AM, "Michael A. Labriola" <la...@digitalprimates.net>
wrote:
>>Makes sense and we probably should have done that in the first place.
>>But since we didn't, do we change behavior and risk breaking folks or
>>add a flag and keep both code paths?
>
>The problem I have with two code paths is how do we choose between them?
>Are we going to do a version number check or make someone compile
>differently, etc.? Also, FWIW as a data point, other than a test which
>was expecting a specific error to be thrown, I am going to highly doubt
>anyone would even know a change was made unless they were working around
>it before. Willing to go either direction but I am always hesitant to be
>dragged down by complete backward compatibility, especially for low risk
>items.
Well, if you want to take the risk, go with the single code path and
comment out the mustella tests and if it breaks someone I will point them
to you. I won't veto such a change.
If it were me, I'd add the flag, default to new behavior, and set the flag
in the mustella tests.
I agree we don't need to be completely backward compatible for past
incorrect behavior, but I'm often reminded of how we think we won't break
anybody and then do.
-Alex
RE: [Mustella] still failing, must fix
Posted by "Michael A. Labriola" <la...@digitalprimates.net>.
>Makes sense and we probably should have done that in the first place. But since we didn't, do we change behavior and risk breaking folks or add a flag and keep both code paths?
The problem I have with two code paths is how do we choose between them? Are we going to do a version number check or make someone compile differently, etc.? Also, FWIW as a data point, other than a test which was expecting a specific error to be thrown, I am going to highly doubt anyone would even know a change was made unless they were working around it before. Willing to go either direction but I am always hesitant to be dragged down by complete backward compatibility, especially for low risk items.
Mike
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
On 6/2/14 10:46 AM, "Michael A. Labriola" <la...@digitalprimates.net>
wrote:
>>A totally reasonable scenario. It is tempting to simply remove this
>>check, but you never know when someone is relying on this behavior.
>
>I am not really suggesting that we remove it completely, just rather that
>we consider it an OR. A SortField's name must either be valid or it must
>have a compareFunction.
Makes sense and we probably should have done that in the first place. But
since we didn't, do we change behavior and risk breaking folks or add a
flag and keep both code paths?
-Alex
RE: [Mustella] still failing, must fix
Posted by "Michael A. Labriola" <la...@digitalprimates.net>.
>A totally reasonable scenario. It is tempting to simply remove this check, but you never know when someone is relying on this behavior.
I am not really suggesting that we remove it completely, just rather that we consider it an OR. A SortField's name must either be valid or it must have a compareFunction.
Mike
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
On 6/2/14 7:38 AM, "Michael A. Labriola" <la...@digitalprimates.net>
wrote:
>>It's a valid question. There is probably no one right answer. My
>>answer is that, if you didn't intend the compareFunction to use the
>>field name in the SortField, why use a SortField at all? Why not just
>>use the top-level Sort compareFunction.
>
>Just continuing the discussion. I think the case here is that you have
>(n) fields but only one of them needed a compareFuntion. This is often
>the case in DataGrids where you have a bunch of standard columns that map
>to an object in the dataprovider and one that is derived.
A totally reasonable scenario. It is tempting to simply remove this
check, but you never know when someone is relying on this behavior.
-Alex
RE: [Mustella] still failing, must fix
Posted by "Michael A. Labriola" <la...@digitalprimates.net>.
>It's a valid question. There is probably no one right answer. My answer is that, if you didn't intend the compareFunction to use the field name in the SortField, why use a SortField at all? Why not just use the top-level Sort compareFunction.
Just continuing the discussion. I think the case here is that you have (n) fields but only one of them needed a compareFuntion. This is often the case in DataGrids where you have a bunch of standard columns that map to an object in the dataprovider and one that is derived.
Mike
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
On 6/1/14 6:01 PM, "Michael A. Labriola" <la...@digitalprimates.net>
wrote:
>>>>You should get an arg count mismatch. By changing the
>>>>(!hasFieldName) test on Sort.as line 413 I got the expected result
>>>>("Find criteria must contain all sort fields")
>>>
>
>Okay, so this is where I have an issue that I am not sure how we should
>resolve. I don't agree with the expected result.
>
>If I provide a custom sort function for a SortField, then why should Sort
>use the 'name' of the ISortField to determine whether or not this
>particular SortField is valid? If something has a SortField with a custom
>sort function, why should we believe that we know more about what the
>user intended and assume that there sort even uses the 'name' property
>for the SortField. The custom sort could be using multiple fields or, for
>all we care, derived from a completely different source. Why would we
>decide if its valid or not based on whether the 'name' can be simply
>indexed into the data provider's object?
It's a valid question. There is probably no one right answer. My answer
is that, if you didn't intend the compareFunction to use the field name in
the SortField, why use a SortField at all? Why not just use the top-level
Sort compareFunction.
Anyway, as I suggested to Nick and his XMLListCollection change, the
answer might just be to add a flag or two so folks can have old behavior
or new behavior.
-Alex
RE: [Mustella] still failing, must fix
Posted by "Michael A. Labriola" <la...@digitalprimates.net>.
>>>You should get an arg count mismatch. By changing the
>>>(!hasFieldName) test on Sort.as line 413 I got the expected result
>>>("Find criteria must contain all sort fields")
>>
Okay, so this is where I have an issue that I am not sure how we should resolve. I don't agree with the expected result.
If I provide a custom sort function for a SortField, then why should Sort use the 'name' of the ISortField to determine whether or not this particular SortField is valid? If something has a SortField with a custom sort function, why should we believe that we know more about what the user intended and assume that there sort even uses the 'name' property for the SortField. The custom sort could be using multiple fields or, for all we care, derived from a completely different source. Why would we decide if its valid or not based on whether the 'name' can be simply indexed into the data provider's object?
Mike
RE: [Mustella] still failing, must fix
Posted by "Michael A. Labriola" <la...@digitalprimates.net>.
>- The Menu/MenuBar/Tree/DataGrid/MXItemRenderer failures are a result of the XMLListCollection change. I looked at MXItemRenderer and it appears that the test is expecting an updateComplete when an XML node is added to the collection. Does this no longer happen because the parent is not notified? I haven't looked any deeper.
I think this is likely due to the way XMLListAdapter handled updates. Basically, it observed the XML for changes and then dispatched update events. I am guessing we are getting out of sync with what's being watched with the new way that source is reset. This is just a hunch, but an educated one.
Mike
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
And in case you missed it, yesterday I found an problem with the commit
for "FLEX-34078 fix callouts closing". I'm assuming we want to get all of
these cleaned up before we do 4.12.2? I probably have a few days of
install script work ahead of me to try to checksum downloads and handle
caching in ant.
-Alex
On 5/27/14 3:07 PM, "Alex Harui" <ah...@adobe.com> wrote:
>And finally:
>
>- The Menu/MenuBar/Tree/DataGrid/MXItemRenderer failures are a result of
>the XMLListCollection change. I looked at MXItemRenderer and it appears
>that the test is expecting an updateComplete when an XML node is added to
>the collection. Does this no longer happen because the parent is not
>notified? I haven't looked any deeper.
>
>- The ComboBox failures are due to the change "FLEX-34222 fix selection
>reverting to previous typed values when second value (not in list) is
>entered". I have not looked into why.
>
>I am now going to return to my regularly scheduled programmingŠ
>
>-Alex
>
>On 5/27/14 1:26 PM, "Alex Harui" <ah...@adobe.com> wrote:
>
>>I just looked at another failure (DataGroup). It also points at this set
>>of changes. It turns out that Sort.fields can be null if you don't have
>>any SortFields and only a Sort.compareFunction. It fails on line 395.
>>You'll need a null check there.
>>
>>On 5/27/14 1:16 PM, "Michael A. Labriola" <la...@digitalprimates.net>
>>wrote:
>>
>>>>You should get an arg count mismatch. By changing the (!hasFieldName)
>>>>test on Sort.as line 413 I got the expected result ("Find criteria must
>>>>contain all sort fields")
>>>
>>>Thanks. I think there may have been a bad merge on my part too as it
>>>seems the logic that was checked in is also missing another condition.
>>>Will try to get this resolved today.
>>>
>>>My concern on the change above is only that a custom sort function could
>>>be on use a label function or something other than a single field, so we
>>>need to ensure that we don't exclusively use the existence of the fields
>>>as an indicator. The original bug was also that rows were being inserted
>>>in the wrong place when using a custom sort function as it was using the
>>>field to determine where, not the function.
>>>
>>>Mike
>>>
>>
>
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
And finally:
- The Menu/MenuBar/Tree/DataGrid/MXItemRenderer failures are a result of
the XMLListCollection change. I looked at MXItemRenderer and it appears
that the test is expecting an updateComplete when an XML node is added to
the collection. Does this no longer happen because the parent is not
notified? I haven't looked any deeper.
- The ComboBox failures are due to the change "FLEX-34222 fix selection
reverting to previous typed values when second value (not in list) is
entered". I have not looked into why.
I am now going to return to my regularly scheduled programmingŠ
-Alex
On 5/27/14 1:26 PM, "Alex Harui" <ah...@adobe.com> wrote:
>I just looked at another failure (DataGroup). It also points at this set
>of changes. It turns out that Sort.fields can be null if you don't have
>any SortFields and only a Sort.compareFunction. It fails on line 395.
>You'll need a null check there.
>
>On 5/27/14 1:16 PM, "Michael A. Labriola" <la...@digitalprimates.net>
>wrote:
>
>>>You should get an arg count mismatch. By changing the (!hasFieldName)
>>>test on Sort.as line 413 I got the expected result ("Find criteria must
>>>contain all sort fields")
>>
>>Thanks. I think there may have been a bad merge on my part too as it
>>seems the logic that was checked in is also missing another condition.
>>Will try to get this resolved today.
>>
>>My concern on the change above is only that a custom sort function could
>>be on use a label function or something other than a single field, so we
>>need to ensure that we don't exclusively use the existence of the fields
>>as an indicator. The original bug was also that rows were being inserted
>>in the wrong place when using a custom sort function as it was using the
>>field to determine where, not the function.
>>
>>Mike
>>
>
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
On 5/30/14 1:59 PM, "Nicholas Kwiatkowski" <ni...@spoon.as> wrote:
>I'm using Windows 8.1, x86 for my machine that is running my mustella
>tests.
I haven't used Win8 yet. I wonder if there is some Control Panel Display
setting that affects the bitmaps.
>
>The reason why it didn't get caught on my local machine is that the
>menubar
>(and related) weren't in the test path for XMLListCollection (as
>determined
>by MustellaDependancyDB). I rarely run the full/full tests on my local
>machine for every small change.
Are there .lnk.xml files in the MenuBar's SWF folder? In theory that's
what MustallaDependencyDB uses to generate the DB and MustellaTestChooser
uses to pick out a set of tests. The link.xml for me does have
XMLListCollection in it. Maybe you've never built tests in MenuBar so the
.lnk.xml files were never generated.
Thanks,
-Alex
Re: [Mustella] still failing, must fix
Posted by Nicholas Kwiatkowski <ni...@spoon.as>.
I'm using Windows 8.1, x86 for my machine that is running my mustella tests.
The reason why it didn't get caught on my local machine is that the menubar
(and related) weren't in the test path for XMLListCollection (as determined
by MustellaDependancyDB). I rarely run the full/full tests on my local
machine for every small change.
I'll see what I can find. Thanks.
On Fri, May 30, 2014 at 4:50 PM, Alex Harui <ah...@adobe.com> wrote:
> A few replies back I pointed out which commits broke what. At this point
> I'm not planning to spend time on trying to make tests pass, so if you
> have cycles to look at the impact of the XMLListCollection change, that
> would be great. In theory you should be seeing different mustella results
> with and without your changes on your local computer. You might get
> bitmap compare failures either way since you said you've had problems
> getting them to pass, but the bad.png files should be different. If
> that's not the case, let's see if we can figure out why.
>
> BTW, regarding your local bitmap compare problem, what OS are you using?
> IIRC some versions of Windows had Control Panel settings that affected
> mustella even with embedded fonts.
>
> -Alex
>
> On 5/30/14 1:30 PM, "Nicholas Kwiatkowski" <ni...@spoon.as> wrote:
>
> >Ok. I'll take another look at the menubar issue as well.
> >
> >
> >On Fri, May 30, 2014 at 4:22 PM, Justin Mclean <ju...@classsoftware.com>
> >wrote:
> >
> >> Hi,
> >>
> >> > So, have the tests been doing any better? I've been watching Erik's
> >> > server, and the last few tests have been failing because of git/server
> >> > issues, not code.
> >>
> >> I've fixed the Alert issues, but Mikes issues and a few other still
> >>exist
> >> I not had time to look further as I'm at a conference and had two 75 min
> >> talks to give today. I may get some time over the weekend/early next
> >>week
> >> but I'm attending another conference next week.
> >>
> >> Thanks,
> >> Justin
>
>
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
A few replies back I pointed out which commits broke what. At this point
I'm not planning to spend time on trying to make tests pass, so if you
have cycles to look at the impact of the XMLListCollection change, that
would be great. In theory you should be seeing different mustella results
with and without your changes on your local computer. You might get
bitmap compare failures either way since you said you've had problems
getting them to pass, but the bad.png files should be different. If
that's not the case, let's see if we can figure out why.
BTW, regarding your local bitmap compare problem, what OS are you using?
IIRC some versions of Windows had Control Panel settings that affected
mustella even with embedded fonts.
-Alex
On 5/30/14 1:30 PM, "Nicholas Kwiatkowski" <ni...@spoon.as> wrote:
>Ok. I'll take another look at the menubar issue as well.
>
>
>On Fri, May 30, 2014 at 4:22 PM, Justin Mclean <ju...@classsoftware.com>
>wrote:
>
>> Hi,
>>
>> > So, have the tests been doing any better? I've been watching Erik's
>> > server, and the last few tests have been failing because of git/server
>> > issues, not code.
>>
>> I've fixed the Alert issues, but Mikes issues and a few other still
>>exist
>> I not had time to look further as I'm at a conference and had two 75 min
>> talks to give today. I may get some time over the weekend/early next
>>week
>> but I'm attending another conference next week.
>>
>> Thanks,
>> Justin
Re: [Mustella] still failing, must fix
Posted by Nicholas Kwiatkowski <ni...@spoon.as>.
Ok. I'll take another look at the menubar issue as well.
On Fri, May 30, 2014 at 4:22 PM, Justin Mclean <ju...@classsoftware.com>
wrote:
> Hi,
>
> > So, have the tests been doing any better? I've been watching Erik's
> > server, and the last few tests have been failing because of git/server
> > issues, not code.
>
> I've fixed the Alert issues, but Mikes issues and a few other still exist
> I not had time to look further as I'm at a conference and had two 75 min
> talks to give today. I may get some time over the weekend/early next week
> but I'm attending another conference next week.
>
> Thanks,
> Justin
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> So, have the tests been doing any better? I've been watching Erik's
> server, and the last few tests have been failing because of git/server
> issues, not code.
I've fixed the Alert issues, but Mikes issues and a few other still exist I not had time to look further as I'm at a conference and had two 75 min talks to give today. I may get some time over the weekend/early next week but I'm attending another conference next week.
Thanks,
Justin
Re: [Mustella] still failing, must fix
Posted by Nicholas Kwiatkowski <ni...@spoon.as>.
So, have the tests been doing any better? I've been watching Erik's
server, and the last few tests have been failing because of git/server
issues, not code.
-Nick
On Tue, May 27, 2014 at 7:25 PM, Michael A. Labriola <
labriola@digitalprimates.net> wrote:
> >I just looked at another failure (DataGroup). It also points at this set
> of changes. It turns out that Sort.fields can be null if you don't have
> any SortFields and only a Sort.compareFunction. It >fails on line 395.
> >You'll need a null check there.
>
> Thanks, that is the one I actually failed to merge so I have a fix for
> this issue, not sure how I didn't manage to commit it, but still looking at
> the other part.
>
> Mike
>
RE: [Mustella] still failing, must fix
Posted by "Michael A. Labriola" <la...@digitalprimates.net>.
>I just looked at another failure (DataGroup). It also points at this set of changes. It turns out that Sort.fields can be null if you don't have any SortFields and only a Sort.compareFunction. It >fails on line 395.
>You'll need a null check there.
Thanks, that is the one I actually failed to merge so I have a fix for this issue, not sure how I didn't manage to commit it, but still looking at the other part.
Mike
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
I just looked at another failure (DataGroup). It also points at this set
of changes. It turns out that Sort.fields can be null if you don't have
any SortFields and only a Sort.compareFunction. It fails on line 395.
You'll need a null check there.
On 5/27/14 1:16 PM, "Michael A. Labriola" <la...@digitalprimates.net>
wrote:
>>You should get an arg count mismatch. By changing the (!hasFieldName)
>>test on Sort.as line 413 I got the expected result ("Find criteria must
>>contain all sort fields")
>
>Thanks. I think there may have been a bad merge on my part too as it
>seems the logic that was checked in is also missing another condition.
>Will try to get this resolved today.
>
>My concern on the change above is only that a custom sort function could
>be on use a label function or something other than a single field, so we
>need to ensure that we don't exclusively use the existence of the fields
>as an indicator. The original bug was also that rows were being inserted
>in the wrong place when using a custom sort function as it was using the
>field to determine where, not the function.
>
>Mike
>
RE: [Mustella] still failing, must fix
Posted by "Michael A. Labriola" <la...@digitalprimates.net>.
>You should get an arg count mismatch. By changing the (!hasFieldName) test on Sort.as line 413 I got the expected result ("Find criteria must contain all sort fields")
Thanks. I think there may have been a bad merge on my part too as it seems the logic that was checked in is also missing another condition. Will try to get this resolved today.
My concern on the change above is only that a custom sort function could be on use a label function or something other than a single field, so we need to ensure that we don't exclusively use the existence of the fields as an indicator. The original bug was also that rows were being inserted in the wrong place when using a custom sort function as it was using the field to determine where, not the function.
Mike
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
This is Justin's test case that replicates the problem in English.
<?xml version="1.0" encoding="utf-8"?>
<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
xmlns:s="library://ns.adobe.com/flex/spark"
xmlns:mx="library://ns.adobe.com/flex/mx"
initialize="triggerFindConditionRTE()">
<fx:Script>
<![CDATA[
import mx.collections.Sort;
import mx.collections.SortField;
private function triggerFindConditionRTE():void{
var a:Array = new Array();
var s:Sort = new Sort();
a.push("a");
s.compareFunction = cmpFn;
var sfArray:Array = new Array();
sfArray.push(new SortField("0"));
sfArray.push(new SortField("field1"));
var o:Object = new Object();
o["0"] = undefined;
o["field1"] = 2;
s.fields = sfArray;
s.findItem(a, o, null);
}
// WTF?
private function cmpFn(o1:Object, o2:Object):void{
}
]]>
</fx:Script>
</s:Application>
You should get an arg count mismatch. By changing the (!hasFieldName)
test on Sort.as line 413 I got the expected result ("Find criteria must
contain all sort fields")
On 5/27/14 12:53 PM, "Michael A. Labriola" <la...@digitalprimates.net>
wrote:
>>Mike, your thoughts on the logic? The test sets up two SortFields and a
>>compare function on the Sort (not the SortFields) then calls findItem.
>The first SortField's field name is a non-existent field. I think the
>old logic would see if the field existed in the data item. The new logic
>seems to assume the fieldName exists as long as there is no SortField
>compare function and skips the check if the field exists in the data.
>
>
>Let me setup and replicate this scenario and mine locally and see what I
>can do.
>
>Mike
RE: [Mustella] still failing, must fix
Posted by "Michael A. Labriola" <la...@digitalprimates.net>.
>Mike, your thoughts on the logic? The test sets up two SortFields and a compare function on the Sort (not the SortFields) then calls findItem.
The first SortField's field name is a non-existent field. I think the old logic would see if the field existed in the data item. The new logic seems to assume the fieldName exists as long as there is no SortField compare function and skips the check if the field exists in the data.
Let me setup and replicate this scenario and mine locally and see what I can do.
Mike
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
The JA langpack tests seem to be broken my Labriola's fix for FLEX-34320.
The new logic around setting the hasFieldName and hadPreviousFieldName
changed and is no longer trigging the expected error which lets the code
continue and trigger this new error.
Mike, your thoughts on the logic? The test sets up two SortFields and a
compare function on the Sort (not the SortFields) then calls findItem.
The first SortField's field name is a non-existent field. I think the old
logic would see if the field existed in the data item. The new logic
seems to assume the fieldName exists as long as there is no SortField
compare function and skips the check if the field exists in the data.
-Alex
On 5/27/14 12:27 PM, "Alex Harui" <ah...@adobe.com> wrote:
>These particular tests are setup up with a try/catch block to trap a
>different error but end up trapping this one instead.
>
>-Alex
>
>On 5/27/14 8:51 AM, "Justin Mclean" <ju...@classsoftware.com> wrote:
>
>>Hi,
>>
>>Looking at the flash log on running the test I can see this error - not
>>sure why I don;t get this if I manually run the test via the code.
>>
>>Error #1063: Argument count mismatch on runtimeErrorTests/cmpFn().
>>Expected 2, got 3.
>>
>>Any one have a n idea why this may be the case?
>>
>>Thanks,
>>Justin
>
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> FB can cache SWCs. I always close and re-open FB when changing SWCs in an
> SDK. Could that somehow affect the different results you got on some
> other mustella tests?
Shouldn't do as those were via via the command line not from FB.
justin
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
FB can cache SWCs. I always close and re-open FB when changing SWCs in an
SDK. Could that somehow affect the different results you got on some
other mustella tests? I got failures in MXItemRenderer and DataGroup.
I'm going to take a look at those now.
On 5/27/14 12:58 PM, "Justin Mclean" <ju...@classsoftware.com> wrote:
>Hi,
>
>> Could you be linking against old SWCs? I'm definitely getting the
>>"number
>> of arguments" error.
>
>Grrrrr looks like FB cached the swcs or something (build number issue?).
>A clean "fixed" the issue.
>
>The app I posed below does get the wrong RTE if you use the latest
>develop branch so at least it should be easy to replicate and debug
>without using mustella.
>
>Thanks,
>Justin
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> Could you be linking against old SWCs? I'm definitely getting the "number
> of arguments" error.
Grrrrr looks like FB cached the swcs or something (build number issue?). A clean "fixed" the issue.
The app I posed below does get the wrong RTE if you use the latest develop branch so at least it should be easy to replicate and debug without using mustella.
Thanks,
Justin
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
Could you be linking against old SWCs? I'm definitely getting the "number
of arguments" error.
-Alex
On 5/27/14 12:38 PM, "Justin Mclean" <ju...@classsoftware.com> wrote:
>Hi,
>
>> These particular tests are setup up with a try/catch block to trap a
>> different error but end up trapping this one instead.
>
>Sure but why does the stand alone code which replicates the test (posed
>in this thread) get the correct exception? Sort of implies there some
>sort of state going on but looking at the test I can't see where as it
>creates a new array and new sort every time.
>
>Justin
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> These particular tests are setup up with a try/catch block to trap a
> different error but end up trapping this one instead.
Sure but why does the stand alone code which replicates the test (posed in this thread) get the correct exception? Sort of implies there some sort of state going on but looking at the test I can't see where as it creates a new array and new sort every time.
Justin
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
These particular tests are setup up with a try/catch block to trap a
different error but end up trapping this one instead.
-Alex
On 5/27/14 8:51 AM, "Justin Mclean" <ju...@classsoftware.com> wrote:
>Hi,
>
>Looking at the flash log on running the test I can see this error - not
>sure why I don;t get this if I manually run the test via the code.
>
>Error #1063: Argument count mismatch on runtimeErrorTests/cmpFn().
>Expected 2, got 3.
>
>Any one have a n idea why this may be the case?
>
>Thanks,
>Justin
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
Looking at the flash log on running the test I can see this error - not sure why I don;t get this if I manually run the test via the code.
Error #1063: Argument count mismatch on runtimeErrorTests/cmpFn(). Expected 2, got 3.
Any one have a n idea why this may be the case?
Thanks,
Justin
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
Running one of the Japanese tests manually with this code:
<?xml version="1.0" encoding="utf-8"?>
<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
xmlns:s="library://ns.adobe.com/flex/spark"
xmlns:mx="library://ns.adobe.com/flex/mx" initialize="triggerFindConditionRTE()">
<fx:Script>
<![CDATA[
import mx.collections.Sort;
import mx.collections.SortField;
private function triggerFindConditionRTE():void{
var a:Array = new Array();
var s:Sort = new Sort();
a.push("a");
s.compareFunction = cmpFn;
var sfArray:Array = new Array();
sfArray.push(new SortField("0"));
sfArray.push(new SortField("field1"));
var o:Object = new Object();
o["0"] = undefined;
o["field1"] = 2;
s.fields = sfArray;
s.findItem(a, o, null);
}
// WTF?
private function cmpFn(o1:Object, o2:Object):void{
}
]]>
</fx:Script>
</s:Application>
Not sure why the compare function is blank but given the test is checking for a RTE it's probably not an issue.
I get message in the exception being:
検索基準には、'field1' にいたるすべてのソートフィールドが含まれている必要があります。
The test is comparing the error with this
検索基準には、'field1' にいたるすべてのソートフィールドが含まれている必要があります。
Can anyone see any issue here? Looks to me as this test should pass:
<TestCase testID="JA_RTE_Sort_FindCondition" description="RTE Tests" keywords="[Runtime Localization,Sort,FindCondition]" >
<body>
<AssertMethodValue method="try { triggerFindConditionRTE(); } catch (e:Error) { value = e.message }" value="検索基準には、'field1' にいたるすべてのソートフィールドが含まれている必要があります。" />
</body>
</TestCase>
Thanks,
Justin
Re: [Mustella] still failing, must fix
Posted by Erik de Bruin <er...@ixsoftware.nl>.
"We" are 99.9% sure, as there is nothing changed on the VM since May 15th,
the last day all the SDK tests passed.
EdB
On Tue, May 27, 2014 at 2:07 PM, Justin Mclean <ju...@classsoftware.com>wrote:
> Hi,
>
> Are we 100% sure the build machine is set up correctly?
>
> I'm not seeingany failures on first run in:
> components/MenuBar/Halo
> components/Menu/Halo
> components/Menu/Spark
> components/Tree
> gumbo/components/ComboBox
>
> I had one failure in these which when rerun passed.
> gumbo/components/DataGrid
> gumbo/components/ListDragDrop
>
> Justin
>
>
>
--
Ix Multimedia Software
Jan Luykenstraat 27
3521 VB Utrecht
T. 06-51952295
I. www.ixsoftware.nl
Re: [Mustella] still failing, must fix
Posted by Alex Harui <ah...@adobe.com>.
Some tests require the full folder to be run all together. Cherry picking
individual tests may not work. It shouldn't be this way, but sometimes
there are other tests that affect the position of components in a
subsequent test and the gradient can be slightly different and cause
problems.
-Alex
On 5/27/14 5:07 AM, "Justin Mclean" <ju...@classsoftware.com> wrote:
>Hi,
>
>Are we 100% sure the build machine is set up correctly?
>
>I'm not seeingany failures on first run in:
>components/MenuBar/Halo
>components/Menu/Halo
>components/Menu/Spark
>components/Tree
>gumbo/components/ComboBox
>
>I had one failure in these which when rerun passed.
>gumbo/components/DataGrid
>gumbo/components/ListDragDrop
>
>Justin
>
>
Re: [Mustella] still failing, must fix
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
Are we 100% sure the build machine is set up correctly?
I'm not seeingany failures on first run in:
components/MenuBar/Halo
components/Menu/Halo
components/Menu/Spark
components/Tree
gumbo/components/ComboBox
I had one failure in these which when rerun passed.
gumbo/components/DataGrid
gumbo/components/ListDragDrop
Justin
Re: [Mustella] still failing, must fix
Posted by Erik de Bruin <er...@ixsoftware.nl>.
Ok, one week later, there are 43 (!) tests failing:
http://flex-mustella.cloudapp.net/job/flex-sdk_mustella/926/consoleText
If not all of the SDK committers from the past month or so take their
responsibility and fix/revert, I'll start and veto the commits in reverse
order, until we pass all Mustella tests again.
Thanks,
EdB
On Thu, May 22, 2014 at 8:48 AM, Erik de Bruin <er...@ixsoftware.nl> wrote:
> Hi,
>
> With the commit rate to the SDK picking up, it won't do to ignore the
> failing tests. Several commits seem to have broken something, and it's
> getting harder to tell which caused what...
>
> I'm calling on all recent committers to look at the changes they submitted
> and please fix what's causing the broken tests!
>
> Also, as a remark to knowingly submitting code that will cause Mustella to
> fail, please don't. Create a branch, run the tests on that and discuss any
> fixes to the tests in relation to that branch. We want to keep 'develop' as
> stable as we can, specifically to avoid situations like the one we're in
> now.
>
> Thanks,
>
> EdB
>
>
>
> --
> Ix Multimedia Software
>
> Jan Luykenstraat 27
> 3521 VB Utrecht
>
> T. 06-51952295
> I. www.ixsoftware.nl
>
--
Ix Multimedia Software
Jan Luykenstraat 27
3521 VB Utrecht
T. 06-51952295
I. www.ixsoftware.nl