You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flex.apache.org by Justin Mclean <ju...@classsoftware.com> on 2012/03/28 03:42:36 UTC
While we're waiting for Mustella
Hi,
I didn't realise it (and don;t think if it's been posted to the list) but the Flex OS SDK does include Mustella and Mustella tests.
These are likely to be out of date and haven't been donated yet (etc etc) just pointing out that you might want to take a look to see how it works/what the tests cover.
http://opensource.adobe.com/svn/opensource/flex/sdk/branches/4.0.0/mustella/
http://opensource.adobe.com/svn/opensource/flex/sdk/branches/4.0.0/frameworks/tests/
Thanks,
Justin
RE: While we're waiting for Mustella
Posted by "Michael A. Labriola" <la...@digitalprimates.net>.
>Might be able to get these from elsewhere? Are these related to flexunit in any way does anyone know?
>>I will check. Guessing they are the .9 versions
So, the answer is yes. It seems like this is a specially built version of FlexUnit .9. I say 'specially built' as it doesn't match the signature of any other version I have saved on my drive. I have not actually diffed all of the files.
Mike
RE: While we're waiting for Mustella
Posted by "Michael A. Labriola" <la...@digitalprimates.net>.
Might be able to get these from elsewhere? Are these related to flexunit in any way does anyone know?
>I will check. Guessing they are the .9 versions
Re: While we're waiting for Mustella
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> It will just mean you haven't broken something obvious.
Which is still better than not knowing you have broken something obvious. :-)
> For example, a ComboBox tests just proves you can pick the second thing on the dropdown. It
> doesn't check all the other ways to interact with a ComboBox.
If you were going to make changes to ComboBox at the moment I think you would need to add some flex unit tests to show that your changes work or alternatively wait and see what test coverage of ComboBox the Mustella tests provide.
I'm aware that these tests are of a limited nature but right now but it's all we have access to. Having some tests means we to try out a few things right now and not wait around to start coding after Mustella is fully donated.
Thanks,
Justin
Re: While we're waiting for Mustella
Posted by Alex Harui <ah...@adobe.com>.
On 3/29/12 3:37 PM, "Justin Mclean" <ju...@classsoftware.com> wrote:
>
>> Also note: the tests only cover a small fraction of the code in the
>> framework. BasicTests is just a quick sanity check of the top feature of
>> every component.
> A sanity check is far better than no tests at all. Means it is possible to try
> out a few structural changes and the like right now and have some confidence
> that you've not broken the SDK.
It will just mean you haven't broken something obvious. For example, a
ComboBox tests just proves you can pick the second thing on the dropdown. It
doesn't check all the other ways to interact with a ComboBox.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: While we're waiting for Mustella
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> I just don't think we should have a dependency on something undocumented and unsupported.
Which I wasn't suggesting. It can probably be changed to flex unit anyway.
> If you want to take a stab at writing tests with Mustella, especially for
> new components
I'd be happy to do so but don't have access to Mustella in it's current form or any documentation. I can take a guess based on the basic tests but you've said these tests are not representative of how the current tests are written. Is there any other public information on Mustella that I'm not aware of?
> Also note: the tests only cover a small fraction of the code in the
> framework. BasicTests is just a quick sanity check of the top feature of
> every component.
A sanity check is far better than no tests at all. Means it is possible to try out a few structural changes and the like right now and have some confidence that you've not broken the SDK. hopefully by the time someone has something to check in the Mustella tests will have been donated.
Thanks,
Justin
Re: While we're waiting for Mustella
Posted by Alex Harui <ah...@adobe.com>.
On 3/29/12 2:10 PM, "Justin Mclean" <ju...@classsoftware.com> wrote:
> Take look at framework/tests (in the Open Source SDK not BlazeDS) it contains
> tests for spark and mx components, mobile skins, performance, layout, graphic
> primitives and a little more. A search on "<TestCase" give 800+ hits.
Yes, but MXUnit is not in there. I don't think mustella is in there either
>
>> Wait for mustella before making significant changes to the framework.
> It seems we have a set of tests in from the OpenSourse SDK that we could use
> now. They could be used to give some confidence that changes wont break
> things. These tests have already been released by Adobe under a MPL 1.1
> licence. I'm not suggesting we check these tests into SVN if that's your
> concern.
Like I said, I cannot stop folks from running tests on your computer, but I
will veto tests being checked in using MXUnit. I just don't think we should
have a dependency on something undocumented and unsupported.
If you want to take a stab at writing tests with Mustella, especially for
new components, I won't veto those, but if you are writing tests for the
framework itself you are duplicating work.
Also note: the tests only cover a small fraction of the code in the
framework. BasicTests is just a quick sanity check of the top feature of
every component.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: While we're waiting for Mustella
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> I think you may be confused. The Adobe Flex Open Source SDK does not
> contain this code.
Take look at framework/tests (in the Open Source SDK not BlazeDS) it contains tests for spark and mx components, mobile skins, performance, layout, graphic primitives and a little more. A search on "<TestCase" give 800+ hits.
> You found it in a zip in the BlazeDS repository.
I found the needed swc to run the tests here.
> Wait for mustella before making significant changes to the framework.
It seems we have a set of tests in from the OpenSourse SDK that we could use now. They could be used to give some confidence that changes wont break things. These tests have already been released by Adobe under a MPL 1.1 licence. I'm not suggesting we check these tests into SVN if that's your concern.
> As Labriola suggested a few weeks back, I am trying to get some core
> mustella pieces donated sooner.
Thanks that would certainly help.
Thanks,
Justin
Re: While we're waiting for Mustella
Posted by Alex Harui <ah...@adobe.com>.
On 3/28/12 10:55 PM, "Justin Mclean" <ju...@classsoftware.com> wrote:
> Hi,
>
>> ComponentSmoke is obsolete.
> Sure when Mustella gets donated.
No, it is obsolete now, as in, none of the Flex developers used it to check
the work for the past several years.
>
>> MXUnit will probably not get donated.
> We can probably use FlexUnit in it place I think. (Not that I've tried.)
>
>> Mustella is supposed to replace it. Please do not proliferate any tests
>> using MXUnit.
>
> Currently we don't have any easy way to test the framework. Some form of
> testing would help I think.
I am advising against using something (MXUnit) that is undocumented and
unsupported.
>
>> but the patterns in BasicTests are not what you will see in the rest of the
>> mustella tests.
>
> I assume the Mustella donation is likely to be several months away at this
> point?
I hope not.
> If we originally knew it would take this long to get Mustella donated
> we probably would of come up with a way of testing some other way of the
> framework by now.
I doubt in these three months we could have replaced the entire mustella
suite.
>
> Having some way of testing the framework (even if that is an "obsolete" way)
> is essential so that we can test patches etc. The testing framework in the
> Open Source SDK seems to fill that need right at this moment. Sure it's not
> perfect and new tests would probably need to be discarded/converted when
> Mustella is donated but is that such a big issue? In it's current form it can
> give some confidence that changes won't break the framework (especially in
> conjunction with flex unit test). That seems rather useful to me.
I think you may be confused. The Adobe Flex Open Source SDK does not
contain this code. You found it in a zip in the BlazeDS repository. I'm
not sure that zip should be there or contain what it does. In fact, today I
will be examining that file to see if we have to remove or change it instead
of working on other issues.
>
> I'm certainly open to any other suggestions you have on how to test the
> framework.
Wait for mustella before making significant changes to the framework. You
can add new locales and new components all you want until then. If you are
needing to change framework classes to do so, then that is another issue.
>
> Would it be possible to provide some Mustella documentation or test "samples"
> before the full donation so we know what to expect? Only if it doesn't cause
> any delays to the donation process of course.
As Labriola suggested a few weeks back, I am trying to get some core
mustella pieces donated sooner. An additional resource in Adobe is getting
up to speed so he can help us clean up the tests and submit them.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: While we're waiting for Mustella
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> ComponentSmoke is obsolete.
Sure when Mustella gets donated.
> MXUnit will probably not get donated.
We can probably use FlexUnit in it place I think. (Not that I've tried.)
> Mustella is supposed to replace it. Please do not proliferate any tests using MXUnit.
Currently we don't have any easy way to test the framework. Some form of testing would help I think.
> but the patterns in BasicTests are not what you will see in the rest of the mustella tests.
I assume the Mustella donation is likely to be several months away at this point? If we originally knew it would take this long to get Mustella donated we probably would of come up with a way of testing some other way of the framework by now.
Having some way of testing the framework (even if that is an "obsolete" way) is essential so that we can test patches etc. The testing framework in the Open Source SDK seems to fill that need right at this moment. Sure it's not perfect and new tests would probably need to be discarded/converted when Mustella is donated but is that such a big issue? In it's current form it can give some confidence that changes won't break the framework (especially in conjunction with flex unit test). That seems rather useful to me.
I'm certainly open to any other suggestions you have on how to test the framework.
Would it be possible to provide some Mustella documentation or test "samples" before the full donation so we know what to expect? Only if it doesn't cause any delays to the donation process of course.
Thanks,
Justin
Re: While we're waiting for Mustella
Posted by Alex Harui <ah...@adobe.com>.
On 3/28/12 7:59 PM, "Justin Mclean" <ju...@classsoftware.com> wrote:
>
> Note that on a Mac not all of the component smoke tests pass (I think that
> been brought up before on the list), but most of them do and that a good start
> :-) Be great if some one could run this on windows and see what they get.
ComponentSmoke is obsolete.
>
> We now have a way (even if it's not perfect) of testing the SDK - anyone have
> anything they want tested/submitted?
MXUnit will probably not get donated. Mustella is supposed to replace it.
Please do not proliferate any tests using MXUnit.
Again, I can't stop you from doing this, but the patterns in BasicTests are
not what you will see in the rest of the mustella tests.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: While we're waiting for Mustella
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
I took a different approach and was able to run component smoke tests. It's actually not even that hard and no command line involved. The following steps assume you're using Flash Builder.
1. Download the open source SDK.
2. Create a new Flex project in the frameworks test directory.
3. Edit the project and change the SDK from 4.6 to your own version of the Apache 4.6 (assuming that's what you want to test).
4. Edit the project source path and add the componentSmoke and basicTest directory (so package paths are correct).
5. Drag and drop ComponentSmoke.mxml and BasicText.mxml to the src default package.
6. Add mxunit.swc to lib directory (drop and drag again).
7. Right click and run as web application ComponentSmoke.mxml and BasicText.mxml.
8. Fix up the couple of path issues with including as files.
7. Right click and run as web application ComponentSmoke.mxml and BasicText.mxml. They will now run - you might want to edit BasicTest.mxml (see comments).
I assume the same can be done for tests in the other directories. graphics, layout, performance etc etc)
Note that on a Mac not all of the component smoke tests pass (I think that been brought up before on the list), but most of them do and that a good start :-) Be great if some one could run this on windows and see what they get.
We now have a way (even if it's not perfect) of testing the SDK - anyone have anything they want tested/submitted?
Thanks,
Justin
Re: While we're waiting for Mustella
Posted by Alex Harui <ah...@adobe.com>.
There are probably some mxunit based tests in the tests folder, but for
basictests, which uses mustella, it should not need it.
On 3/28/12 2:52 PM, "Justin Mclean" <ju...@classsoftware.com> wrote:
> Hi,
>
>> Flexunit and mxunit should not be required for mustella tests.
>
> The component smoke tests have this:
> import qa.mxunit.*;
>
> I'm assuming it just an old version of fluxunit with a different package name?
> It's likely that the to be donated Mustella tests don't have this issue.
>
>> Are you getting output in the flashlog.txt?
> Not yet no but I've only taken a very quick look at it.
>
> Thanks,
> Justin
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: While we're waiting for Mustella
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> Flexunit and mxunit should not be required for mustella tests.
The component smoke tests have this:
import qa.mxunit.*;
I'm assuming it just an old version of fluxunit with a different package name? It's likely that the to be donated Mustella tests don't have this issue.
> Are you getting output in the flashlog.txt?
Not yet no but I've only taken a very quick look at it.
Thanks,
Justin
Re: While we're waiting for Mustella
Posted by Alex Harui <ah...@adobe.com>.
On 3/28/12 12:59 AM, "Justin Mclean" <ju...@classsoftware.com> wrote:
> 3. Download qa-framworks.zip form here:
> http://opensource.adobe.com/svn/opensource/blazeds/trunk/qa/resources/framewor
> ks/
> Unzip and copy flexunit.swc and mxunit.swc to the mustella directory.
Flexunit and mxunit should not be required for mustella tests.
>
> 6. Run a test by viewing the compiled swf in browser.
> The output should be logged in flashlog.txt and you can then view any errors
> by running MustellaResultParser.java.
>
> Not quite got the last step to work - perhaps my mm.cfg doesn't have the right
> settings.
Mustella results parser is customized to wade through the output of these
tests, so new tests might require tweaking that file.
Are you getting output in the flashlog.txt?
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: While we're waiting for Mustella
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
Results of a little experiment with the OS version of Mustella.
1. Obtain the full open source Adobe Flex SDKs
You can check out from here:
http://opensource.adobe.com/svn/opensource/flex/sdk/trunk
2. Copy the frameworks/test directory and mustella directories into your local Apache Flex SDK directory
3. Download qa-framworks.zip form here:
http://opensource.adobe.com/svn/opensource/blazeds/trunk/qa/resources/frameworks/
Unzip and copy flexunit.swc and mxunit.swc to the mustella directory.
Might be able to get these from elsewhere? Are these related to flexunit in any way does anyone know?
4. Create a shell script called "runtest" in the mustella directory:
java -jar ../lib/mxmlc.jar +flexlib=../frameworks -debug -allow-source-path-overlap --show-actionscript-warnings=true -strict=true --library-path+=../mustella --source-path=../frameworks/projects/framework/src,../frameworks/tests/$1 ../frameworks/tests/$1/$2
5. Compile a test by running runtest script
Pass runtest the directory name and the mxml file you want compiled. For example:
./runtest graphics GraphicTags_checkin.mxml
./runtest basicTests BasicTests.mxml
I had a issue with font paths in the CSS for the BasicTests so just commented it out of the CSS.
6. Run a test by viewing the compiled swf in browser.
The output should be logged in flashlog.txt and you can then view any errors by running MustellaResultParser.java.
Not quite got the last step to work - perhaps my mm.cfg doesn't have the right settings.
Anyone else want to have a go?
Thanks,
Justin
Re: While we're waiting for Mustella
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> That source is still under Adobe copyright.
Totally understand that should not be committed into the whiteboard/branch/trunck etc etc just that they might be useful for people to have a look to have some idea (as opposed to none) of how Mustella works and what the tests may look like.
> The tests in the frameworks/tests folder are not representative of how we
> have written tests.
Sure but having some test is probably better than having none so it might be useful to see if they can be run in some way.
> We are donating a more flexible command-line utility
Also good to know.
Thanks,
Justin
Re: While we're waiting for Mustella
Posted by Alex Harui <ah...@adobe.com>.
On 3/27/12 6:42 PM, "Justin Mclean" <ju...@classsoftware.com> wrote:
> Hi,
>
> I didn't realise it (and don;t think if it's been posted to the list) but the
> Flex OS SDK does include Mustella and Mustella tests.
>
> These are likely to be out of date and haven't been donated yet (etc etc) just
> pointing out that you might want to take a look to see how it works/what the
> tests cover.
>
> http://opensource.adobe.com/svn/opensource/flex/sdk/branches/4.0.0/mustella/
> http://opensource.adobe.com/svn/opensource/flex/sdk/branches/4.0.0/frameworks/
> tests/
>
That source is still under Adobe copyright. I'm not sure you should try to
reverse engineer the swc in the mustella folder there, but it does contain
most of the test steps.
The tests in the frameworks/tests folder are not representative of how we
have written tests. It is a special case to quickly run through the most
common case of all of the components. The tests we want to donate run lots
of tests on one component at a time.
The way the test is built, run and analyzed is also custom. We are donating
a more flexible command-line utility that handles timeouts and output in a
better way.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui