You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flex.apache.org by Michael Schmalle <ap...@teotigraphix.com> on 2012/11/14 18:17:51 UTC
[FALCON] Functional Testing
Hi,
I finally have some time to get back into the deep recesses of the
compiler which I will say is probably the place I can afford to spend
my time.
Now, We have unit tests and SWC functional tests. We do not have MXMLC
functional tests as it stands. Cyrill has a basic template he has
started in his GITHub account which basically mimics the way we do it
with a SWC.
My question is, I can see this getting mammoth with mxml, fxg etc
sample files. How do we go about organizing this so we can test the
functionality that seems broken as we fix it?
I have had thoughts of trying to reduce the Exceptions down to unit
testable artifacts but in the case of the FXGTranscoder exception,
this is impossible to unit test since it needs MXMLC to get to the
Exception.
Any thoughts?
Mike
--
Michael Schmalle - Teoti Graphix, LLC
http://www.teotigraphix.com
http://blog.teotigraphix.com
Re: [FALCON] Functional Testing
Posted by Alex Harui <ah...@adobe.com>.
On 11/14/12 10:42 AM, "Michael Schmalle" <ap...@teotigraphix.com> wrote:
> I will stay out of this discussion but I do agree Gordon that taking a
> more "unit" approach to functional testing would create a stronger
> foundation for reaching levels of stability with the compiler as a
> standalone compiler that uses a specific dialect.
Yup, if the tests are going to be non-visual SWFs then you don't need all of
Mustella. The "checkintests" target in the SDK's build.xml also compiles
and runs a SWF. Yes, it uses Mustella, but really, it doesn't have to. If
you are using ANT to drive the unit tests, you can probably copy the way we
launch the SWF and read the trace log.
>
> This way we could more look at fixing the existing framework based on
> the stricter use of the new compiler.
>
> But again, I will stay out until the heavy weights decide and point
> the direction.
>
I'm not certain, but I'm pretty sure Gordon weighs more than me.
> Mike
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
RE: [FALCON] Functional Testing
Posted by Michael Schmalle <ap...@teotigraphix.com>.
I will stay out of this discussion but I do agree Gordon that taking a
more "unit" approach to functional testing would create a stronger
foundation for reaching levels of stability with the compiler as a
standalone compiler that uses a specific dialect.
This way we could more look at fixing the existing framework based on
the stricter use of the new compiler.
But again, I will stay out until the heavy weights decide and point
the direction.
Mike
Quoting Gordon Smith <go...@adobe.com>:
> I strongly prefer not to use Mustella for functional tests of
> Falcon. Doing so would cause a chicken-and-egg issue of whether
> Falcon can compile Mustella's test language.
>
> I plan to have EXTREMELY simple functional tests. For example, the
> functional test for the <int> tag should only require that Falcon
> understand a document tag (<Sprite>), the <Declararations> tag, and
> the <int> tag.
>
> - Gordon
>
> -----Original Message-----
> From: Alex Harui [mailto:aharui@adobe.com]
> Sent: Wednesday, November 14, 2012 10:34 AM
> To: flex-dev@incubator.apache.org
> Subject: Re: [FALCON] Functional Testing
>
>
>
>
> On 11/14/12 10:29 AM, "Michael Schmalle" <ap...@teotigraphix.com> wrote:
>
>> Ok so we have something basically setup like SWC tests.
>>
>> I will be honest, I have NO IDEA how to do;
>>
>> - run the SWF in Flash Player
>> - get the trace output, and check it
>>
>> If you would set that stuff up sometime and give me a template, I'll
>> get the idea.
> That's what Mustella does. I haven't looked at these functional
> tests, but if the entry point is MXMLC and the output is a runnable
> SWF, then I would think we could just use Mustella for functional
> tests? It would also be a way to cross check what the old MXMLC does.
>>
>
> --
> Alex Harui
> Flex SDK Team
> Adobe Systems, Inc.
> http://blogs.adobe.com/aharui
>
>
--
Michael Schmalle - Teoti Graphix, LLC
http://www.teotigraphix.com
http://blog.teotigraphix.com
RE: [FALCON] Functional Testing
Posted by Gordon Smith <go...@adobe.com>.
I strongly prefer not to use Mustella for functional tests of Falcon. Doing so would cause a chicken-and-egg issue of whether Falcon can compile Mustella's test language.
I plan to have EXTREMELY simple functional tests. For example, the functional test for the <int> tag should only require that Falcon understand a document tag (<Sprite>), the <Declararations> tag, and the <int> tag.
- Gordon
-----Original Message-----
From: Alex Harui [mailto:aharui@adobe.com]
Sent: Wednesday, November 14, 2012 10:34 AM
To: flex-dev@incubator.apache.org
Subject: Re: [FALCON] Functional Testing
On 11/14/12 10:29 AM, "Michael Schmalle" <ap...@teotigraphix.com> wrote:
> Ok so we have something basically setup like SWC tests.
>
> I will be honest, I have NO IDEA how to do;
>
> - run the SWF in Flash Player
> - get the trace output, and check it
>
> If you would set that stuff up sometime and give me a template, I'll
> get the idea.
That's what Mustella does. I haven't looked at these functional tests, but if the entry point is MXMLC and the output is a runnable SWF, then I would think we could just use Mustella for functional tests? It would also be a way to cross check what the old MXMLC does.
>
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
RE: [FALCON] Functional Testing
Posted by Michael Schmalle <ap...@teotigraphix.com>.
Yes!
This is exactly what I was thinking and worked great, thanks Gordon.
Mike
Quoting Gordon Smith <go...@adobe.com>:
>> What I do not get is if there is a way I can back track to where
>> the problem is added so I could start in the code closer to the
>> problem.
>
> Set a breakpoint in the constructor of CompilerProblem, the base
> class for every Falcon class that represents a compiler problem.
>
> - Gordon
>
--
Michael Schmalle - Teoti Graphix, LLC
http://www.teotigraphix.com
http://blog.teotigraphix.com
RE: [FALCON] Functional Testing
Posted by Gordon Smith <go...@adobe.com>.
> What I do not get is if there is a way I can back track to where the problem is added so I could start in the code closer to the problem.
Set a breakpoint in the constructor of CompilerProblem, the base class for every Falcon class that represents a compiler problem.
- Gordon
-----Original Message-----
From: Michael Schmalle [mailto:apache@teotigraphix.com]
Sent: Wednesday, November 14, 2012 10:30 AM
To: flex-dev@incubator.apache.org
Subject: RE: [FALCON] Functional Testing
Ok so we have something basically setup like SWC tests.
I will be honest, I have NO IDEA how to do;
- run the SWF in Flash Player
- get the trace output, and check it
If you would set that stuff up sometime and give me a template, I'll get the idea.
Some of this stuff in the MXML AST is going to take a bit for me to digest. I spent 30 minutes stepping through code to see why AddChild can't have an id. I got to where AddChild as added as a child of the State and it did have an id set in the instance.
What I do not get is if there is a way I can back track to where the problem is added so I could start in the code closer to the problem.
I tried running the spark SWC tests and mx but there are some obvious errors that are going to take me a bit more time to digest where they exists.
Mike
Quoting Gordon Smith <go...@adobe.com>:
> The functional tests should simply call the entry point of MXMLC,
> check that there are no problems, and run the SWF in Flash Player, get
> the trace output, and check it. I also recommend that, as with the
> parsing unit tests I've been writing, you don't have to necessarily
> put the code to be compiled into a separated file. It makes it easy to
> see what the test is testing when you can just look at a String in the
> JUnit test.
>
> - Gordon
>
> -----Original Message-----
> From: Michael Schmalle [mailto:apache@teotigraphix.com]
> Sent: Wednesday, November 14, 2012 9:18 AM
> To: flex-dev@incubator.apache.org
> Subject: [FALCON] Functional Testing
>
> Hi,
>
> I finally have some time to get back into the deep recesses of the
> compiler which I will say is probably the place I can afford to spend
> my time.
>
> Now, We have unit tests and SWC functional tests. We do not have MXMLC
> functional tests as it stands. Cyrill has a basic template he has
> started in his GITHub account which basically mimics the way we do it
> with a SWC.
>
> My question is, I can see this getting mammoth with mxml, fxg etc
> sample files. How do we go about organizing this so we can test the
> functionality that seems broken as we fix it?
>
> I have had thoughts of trying to reduce the Exceptions down to unit
> testable artifacts but in the case of the FXGTranscoder exception,
> this is impossible to unit test since it needs MXMLC to get to the
> Exception.
>
> Any thoughts?
>
> Mike
>
> --
> Michael Schmalle - Teoti Graphix, LLC
> http://www.teotigraphix.com
> http://blog.teotigraphix.com
>
>
--
Michael Schmalle - Teoti Graphix, LLC
http://www.teotigraphix.com
http://blog.teotigraphix.com
RE: [FALCON] Functional Testing
Posted by Gordon Smith <go...@adobe.com>.
I'll work on setting up the first few functional tests.
- Gordon
-----Original Message-----
From: Michael Schmalle [mailto:apache@teotigraphix.com]
Sent: Wednesday, November 14, 2012 10:30 AM
To: flex-dev@incubator.apache.org
Subject: RE: [FALCON] Functional Testing
Ok so we have something basically setup like SWC tests.
I will be honest, I have NO IDEA how to do;
- run the SWF in Flash Player
- get the trace output, and check it
If you would set that stuff up sometime and give me a template, I'll get the idea.
Some of this stuff in the MXML AST is going to take a bit for me to digest. I spent 30 minutes stepping through code to see why AddChild can't have an id. I got to where AddChild as added as a child of the State and it did have an id set in the instance.
What I do not get is if there is a way I can back track to where the problem is added so I could start in the code closer to the problem.
I tried running the spark SWC tests and mx but there are some obvious errors that are going to take me a bit more time to digest where they exists.
Mike
Quoting Gordon Smith <go...@adobe.com>:
> The functional tests should simply call the entry point of MXMLC,
> check that there are no problems, and run the SWF in Flash Player, get
> the trace output, and check it. I also recommend that, as with the
> parsing unit tests I've been writing, you don't have to necessarily
> put the code to be compiled into a separated file. It makes it easy to
> see what the test is testing when you can just look at a String in the
> JUnit test.
>
> - Gordon
>
> -----Original Message-----
> From: Michael Schmalle [mailto:apache@teotigraphix.com]
> Sent: Wednesday, November 14, 2012 9:18 AM
> To: flex-dev@incubator.apache.org
> Subject: [FALCON] Functional Testing
>
> Hi,
>
> I finally have some time to get back into the deep recesses of the
> compiler which I will say is probably the place I can afford to spend
> my time.
>
> Now, We have unit tests and SWC functional tests. We do not have MXMLC
> functional tests as it stands. Cyrill has a basic template he has
> started in his GITHub account which basically mimics the way we do it
> with a SWC.
>
> My question is, I can see this getting mammoth with mxml, fxg etc
> sample files. How do we go about organizing this so we can test the
> functionality that seems broken as we fix it?
>
> I have had thoughts of trying to reduce the Exceptions down to unit
> testable artifacts but in the case of the FXGTranscoder exception,
> this is impossible to unit test since it needs MXMLC to get to the
> Exception.
>
> Any thoughts?
>
> Mike
>
> --
> Michael Schmalle - Teoti Graphix, LLC
> http://www.teotigraphix.com
> http://blog.teotigraphix.com
>
>
--
Michael Schmalle - Teoti Graphix, LLC
http://www.teotigraphix.com
http://blog.teotigraphix.com
Re: [FALCON] Functional Testing
Posted by Michael Schmalle <ap...@teotigraphix.com>.
Yeah I was already thinking this, I only suggested the "now" option so
I could start "getting it" quicker, since I must say, the mustella
scares me. :) It almost ate my computer.
Jokes aside, I am a programmer and kind of hard headed to the dance of
all the systems together. So someone says do, this and that to test
and fix against the mustella, I will be on it.
Mike
Quoting Alex Harui <ah...@adobe.com>:
>
>
>
> On 11/14/12 10:29 AM, "Michael Schmalle" <ap...@teotigraphix.com> wrote:
>
>> Ok so we have something basically setup like SWC tests.
>>
>> I will be honest, I have NO IDEA how to do;
>>
>> - run the SWF in Flash Player
>> - get the trace output, and check it
>>
>> If you would set that stuff up sometime and give me a template, I'll
>> get the idea.
> That's what Mustella does. I haven't looked at these functional tests, but
> if the entry point is MXMLC and the output is a runnable SWF, then I would
> think we could just use Mustella for functional tests? It would also be a
> way to cross check what the old MXMLC does.
>>
>
> --
> Alex Harui
> Flex SDK Team
> Adobe Systems, Inc.
> http://blogs.adobe.com/aharui
>
>
--
Michael Schmalle - Teoti Graphix, LLC
http://www.teotigraphix.com
http://blog.teotigraphix.com
Re: [FALCON] Functional Testing
Posted by Alex Harui <ah...@adobe.com>.
On 11/14/12 10:29 AM, "Michael Schmalle" <ap...@teotigraphix.com> wrote:
> Ok so we have something basically setup like SWC tests.
>
> I will be honest, I have NO IDEA how to do;
>
> - run the SWF in Flash Player
> - get the trace output, and check it
>
> If you would set that stuff up sometime and give me a template, I'll
> get the idea.
That's what Mustella does. I haven't looked at these functional tests, but
if the entry point is MXMLC and the output is a runnable SWF, then I would
think we could just use Mustella for functional tests? It would also be a
way to cross check what the old MXMLC does.
>
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
RE: [FALCON] Functional Testing
Posted by Michael Schmalle <ap...@teotigraphix.com>.
Ok so we have something basically setup like SWC tests.
I will be honest, I have NO IDEA how to do;
- run the SWF in Flash Player
- get the trace output, and check it
If you would set that stuff up sometime and give me a template, I'll
get the idea.
Some of this stuff in the MXML AST is going to take a bit for me to
digest. I spent 30 minutes stepping through code to see why AddChild
can't have an id. I got to where AddChild as added as a child of the
State and it did have an id set in the instance.
What I do not get is if there is a way I can back track to where the
problem is added so I could start in the code closer to the problem.
I tried running the spark SWC tests and mx but there are some obvious
errors that are going to take me a bit more time to digest where they
exists.
Mike
Quoting Gordon Smith <go...@adobe.com>:
> The functional tests should simply call the entry point of MXMLC,
> check that there are no problems, and run the SWF in Flash Player,
> get the trace output, and check it. I also recommend that, as with
> the parsing unit tests I've been writing, you don't have to
> necessarily put the code to be compiled into a separated file. It
> makes it easy to see what the test is testing when you can just look
> at a String in the JUnit test.
>
> - Gordon
>
> -----Original Message-----
> From: Michael Schmalle [mailto:apache@teotigraphix.com]
> Sent: Wednesday, November 14, 2012 9:18 AM
> To: flex-dev@incubator.apache.org
> Subject: [FALCON] Functional Testing
>
> Hi,
>
> I finally have some time to get back into the deep recesses of the
> compiler which I will say is probably the place I can afford to
> spend my time.
>
> Now, We have unit tests and SWC functional tests. We do not have
> MXMLC functional tests as it stands. Cyrill has a basic template he
> has started in his GITHub account which basically mimics the way we
> do it with a SWC.
>
> My question is, I can see this getting mammoth with mxml, fxg etc
> sample files. How do we go about organizing this so we can test the
> functionality that seems broken as we fix it?
>
> I have had thoughts of trying to reduce the Exceptions down to unit
> testable artifacts but in the case of the FXGTranscoder exception,
> this is impossible to unit test since it needs MXMLC to get to the
> Exception.
>
> Any thoughts?
>
> Mike
>
> --
> Michael Schmalle - Teoti Graphix, LLC
> http://www.teotigraphix.com
> http://blog.teotigraphix.com
>
>
--
Michael Schmalle - Teoti Graphix, LLC
http://www.teotigraphix.com
http://blog.teotigraphix.com
RE: [FALCON] Functional Testing
Posted by Gordon Smith <go...@adobe.com>.
The functional tests should simply call the entry point of MXMLC, check that there are no problems, and run the SWF in Flash Player, get the trace output, and check it. I also recommend that, as with the parsing unit tests I've been writing, you don't have to necessarily put the code to be compiled into a separated file. It makes it easy to see what the test is testing when you can just look at a String in the JUnit test.
- Gordon
-----Original Message-----
From: Michael Schmalle [mailto:apache@teotigraphix.com]
Sent: Wednesday, November 14, 2012 9:18 AM
To: flex-dev@incubator.apache.org
Subject: [FALCON] Functional Testing
Hi,
I finally have some time to get back into the deep recesses of the compiler which I will say is probably the place I can afford to spend my time.
Now, We have unit tests and SWC functional tests. We do not have MXMLC functional tests as it stands. Cyrill has a basic template he has started in his GITHub account which basically mimics the way we do it with a SWC.
My question is, I can see this getting mammoth with mxml, fxg etc sample files. How do we go about organizing this so we can test the functionality that seems broken as we fix it?
I have had thoughts of trying to reduce the Exceptions down to unit testable artifacts but in the case of the FXGTranscoder exception, this is impossible to unit test since it needs MXMLC to get to the Exception.
Any thoughts?
Mike
--
Michael Schmalle - Teoti Graphix, LLC
http://www.teotigraphix.com
http://blog.teotigraphix.com