You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flex.apache.org by Om <bi...@gmail.com> on 2013/03/20 02:08:25 UTC
Rendering layer in Flex <-> HTML5/JS solution
Jude,
Sorry for the late response. I am still in the middle of trying out
certain things to first of all, convince myself that this approach is
viable. Responses inline on to thoughts:
On Sun, Mar 17, 2013 at 2:26 PM, jude <fl...@gmail.com> wrote:
> Om,
>
> There's FXG and SVG but there's also HTML5 canvas element.
>
> You probably already know this but FXG is made up of graphic primitives
> such as Rect, Path, Ellipse, Stroke, Fill and BitmapFill and so on. We also
> know that MXML / XML gets converted into ActionScript. In the case of FXG
> it gets converted into graphics commands.
>
Not so with the Falcon compiler. There is no intermediate AS code that
Falcon generates.
>
> So this FXG:
>
> <s:Group>
> <s:Path data="Q 75 75 50 0" width="132" height="150" />
> </s:Group>
> gets converted into something like this the MXMLC compiler:
>
> var sprite:Sprite = new Sprite();
> sprite.width = 132;
> sprite.height = 150;
> sprite.graphics.drawPath(75,75,50,0);
> stage.addChild(sprite);
>
> And that gets converted into something lower level later on. What I was
> mentioning to Alex about is that when we go through the AS to JS
> translation we could translate graphics calls like above to the canvas
> element below:
>
> // Create canvas element
> canvas = document.createElement('canvas');
> canvas.setAttribute('width',132);
> canvas.setAttribute('height',150);
>
> var graphics = canvas.getContext('2d');
> graphics.beginPath();
> graphics.arc(75,75,50,0,Math.PI*2,true); // Outer circle
> document.addElement(canvas);
>
>
This is a totally valid approach. If my FXG -> SVG approach fails, this is
probably I will be trying. But this approach has problems inherently.
Once you draw your graphics on the canvas, you are essentially rasterizing
it which means that we cant easily do skins that scale. Less so, things
like 9-scale skinning, etc.
>
> It's not a 1 to 1 conversion but it's pretty close. Whoever spec'd out the
> canvas element took some notes from the Flash API or took some notes from
> whoever Flash took some notes from. I think we could get a much better
> representation with it than going the current way which is to use the stock
> browser and OS browser component set.
>
Whoever designed FXG took a lot of notes from SVG as well - given that both
were created by Adobe ;-)
>
> The Flex button component is drawn using graphic primitives. We know that's
> always going to look the same because we control what's drawn and where. If
> we just create wrappers around the HTML controls we don't know what we will
> get and we're going to inherit all of HTML's problems. If we draw to the
> canvas what you draw is what you get.
>
I agree that if we use the basic HTML controls, we probably will run into
the approaches you mention. But rather I want to explore how to use SVG,
CSS and JS to attain the rendering fidelity.
Moreover, drawing and redrawing on the Canvas might not be as performant as
we would like. Agreed that we haven't really tested the performance of
SVG, but I am hopeful.
>
> To make this work we'd have to create our own graphics api class in JS that
> was included. It could be really basic and allow use to use Flex skins,
> graphics calls and skins. The downside might be it sacrifices
> accessibility. But if it were up to me and that was the only reason why we
> hesitate to use it I'd say let's output an accessible version at the same
> time. We can do that now. We didn't have that option before. Then screen
> readers will be satisfied and we can avoid a whole mess of problems.
>
SVG supports basic accessibility: http://www.w3.org/TR/SVG-access/ And it
seems that we can make it more accessible by mixing with HTML as needed.
Compared to this, drawing on the Canvas pretty much kills accessibility for
us. It would be much more work to make it accessible.
>
> It was brought up before but if we could choose the abstract out the
> renderer we could probably we might be able to solve a few more problems.
> When I saw Starling running the first time it was like seeing the iPhone
> retina display the first time. It's night and day. Maybe we can have a
> flash display list renderer (current), Starling 2d display list renderer,
> canvas display list renderer, webgl render or some raster bitmap display
> list renderer. These would just translate the original drawing api to the
> target api. Ok I know that's a lot of work and I'm rambling : P
>
I totally agree with you. Getting Flex to render on Starling or even
directly using Stage3D is something I am very much interested in. But,
that is a pure Flash/ActionScript problem. As long as we keep the current
skinning model, we can always create an alternative implementation of FXG
in Flex to use render on the GPU directly.
Please go on with the rambling :-) Its nice to know that others are
thinking about things I am thinking as well - and that I am crazy!
BTW, I have a nice little carefully crafted XSLT that converts a basic FXG
node with just the Rect element to SVG. I am planning on tackling the
'fill' element next. Once I have a decently working XSLT that simply
converts an FXG skin that can be directly rendered on a browser, I will
start posting my work for others to look at at hopefully, contribute.
Thanks,
Om
>
>
> On Thu, Mar 14, 2013 at 5:14 PM, Om <bi...@gmail.com> wrote:
>
> > On Thu, Mar 14, 2013 at 1:24 PM, Alex Harui <ah...@adobe.com> wrote:
> >
> > >
> > >
> > >
> > > On 3/14/13 1:02 PM, "Om" <bi...@gmail.com> wrote:
> > >
> > > >> Don't PhotoShop and Illustrator output SVG as well? What is it
> about
> > > FXG
> > > >> that is a must-have especially if you are targeting HTML and not
> > Flash?
> > > >
> > > >
> > > > This implies that I need to decide on the target (HTML vs. Flash)
> > before
> > > I
> > > > even start designing the skin for the app. Is that what you expect
> > > > developers to do with FlexJS?
> > > Nope, I think they should just choose SVG, and FlexJS and its compiler
> > > should try to convert it into Flash assets when running on Flash.
> >
> >
> > Right, except that when the user chooses the SVG route, that eliminates
> > support for older browsers.
> >
> >
> > > Frankly,
> > > I'm not sure if it has to do a great job in terms of fidelity or
> > > performance. For most folks, the end goal is to get a great HTML/JS
> app.
> > > The SWF version is so you can develop and test as much as possible
> before
> > > cross-compiling.
> > >
> > >
> > If I may suggest an alternative approach, I would use the SWF version to
> > support older browsers. Remember, Flash Player for Desktop is still very
> > prevalent.
> >
> > For the newer browsers that support do support inline SVG, we can convert
> > FXG to SVG and we have a viable non-swf alternative. This is a more
> > future-safe approach, IMHO.
> >
> > >
> > > > My point is that we have tools that create FXG, we have AS code that
> > can
> > > > work with FXG. I believe it is a more efficient approach run with
> FXG
> > > and
> > > > make it work with HTML/JS. The end result would make the SDK users
> > that
> > > > much happier.
> > > The AS code that works with FXG probably uses a lot of Flash APIs, so
> it
> > > can't be cross-compiled efficiently to JS. If you can write an
> efficient
> > > FXG renderer on the JS side, please do so.
> > >
> >
> > No, thats not what I meant. I said "AS code can work *with *FXG". This
> > can be translated to JS code working with SVG. AS to JS translation is
> > what you guys are working on. FXG to SVG XMSLT transformation is
> > (hopefully) the only missing link.
> >
> >
> >
> > > >
> > > > On the flip side, you have not convinced me that we should drop FXG.
> > > I am not trying to convince you to drop FXG, I am just saying that I
> > would
> > > rather write code to support SVG instead and may do so after I get
> bitmap
> > > skinning working. IMO, every year, fewer and fewer new releases of
> tools
> > > will output FXG unless we can show the world a reason it is better than
> > > SVG.
> > >
> > > But again, you or anyone is welcome to write the FXG support, and I
> will
> > > welcome it.
> > >
> >
> > I will hopefully get to work on it sooner than later. I want to put this
> > idea out and let you guys kick the tires to see if I am missing something
> > obvious.
> >
> > Thanks,
> > Om
> >
>
Re: Rendering layer in Flex <-> HTML5/JS solution
Posted by jude <fl...@gmail.com>.
On Tue, Mar 19, 2013 at 8:08 PM, Om <bi...@gmail.com> wrote:
> > To make this work we'd have to create our own graphics api class in JS
> that
> > was included. It could be really basic and allow use to use Flex skins,
> > graphics calls and skins. The downside might be it sacrifices
> > accessibility. But if it were up to me and that was the only reason why
> we
> > hesitate to use it I'd say let's output an accessible version at the same
> > time. We can do that now. We didn't have that option before. Then screen
> > readers will be satisfied and we can avoid a whole mess of problems.
> >
>
>
> SVG supports basic accessibility: http://www.w3.org/TR/SVG-access/ And
> it
> seems that we can make it more accessible by mixing with HTML as needed.
>
> Compared to this, drawing on the Canvas pretty much kills accessibility for
> us. It would be much more work to make it accessible.
>
>
On accessibility. We now have the Falcon compiler so we can create an
accessible version at the same time. For example, at one agency I worked at
we would create a Flash site for our clients and then another static HTML
site for SEO and non-Flash enabled browsers. This was a while ago but it
worked. We could probably make an optimized accessible version better than
an all in one solution because we can throw out everything that's not
needed. There was a thread from about a year ago that described a similar
solution about adding things into the HTML wrapper page and creating a site
map for SEO and deep linking. If Falcon enables it at some point in the
time it is something we can think about doing. Thinking out loud here. ;)
Re: Rendering layer in Flex <-> HTML5/JS solution
Posted by Om <bi...@gmail.com>.
On Wed, Mar 20, 2013 at 9:38 AM, Alex Harui <ah...@adobe.com> wrote:
>
>
>
> On 3/20/13 9:15 AM, "jude" <fl...@gmail.com> wrote:
>
> > On Tue, Mar 19, 2013 at 8:08 PM, Om <bi...@gmail.com> wrote:
> >>
> >>
> >> Please go on with the rambling :-) Its nice to know that others are
> >> thinking about things I am thinking as well - and that I am crazy!
> >>
> >> BTW, I have a nice little carefully crafted XSLT that converts a basic
> FXG
> >> node with just the Rect element to SVG. I am planning on tackling the
> >> 'fill' element next. Once I have a decently working XSLT that simply
> >> converts an FXG skin that can be directly rendered on a browser, I will
> >> start posting my work for others to look at at hopefully, contribute.
> >>
> >
> > Cool. :) I was reading a post yesterday about the massive performance
> > improvements by rasterizing the vectors and then moving them to the GPU,
> > http://goo.gl/WXkmA. I haven't seen this approach before and it may not
> > have been possible before but with Stage 3d, Starling and WebGL it now
> is a
> > choice. And if that works then we may be able to apply it to the Flex SDK
> > and even HTML output (it renders at difference sizing so as you zoom in
> or
> > out it maintains fidelity - I think).
> Did I read that post correctly? I think it supports my choice of bitmaps
> over vectors for the future.
>
The way I see it, bitmaps are good for performance for some scenarios. But
that does not mean that we have to do away with vectors completely. I dont
see a straight line between these two different trains of thoughts.
If I want to have a plain blue colored background on an app thats going to
run on a retina display, why would I chose to embed a big ass image instead
of just drawing a blue colored rectangle (like we do today in the spark/fxg
skins) And what happens if I want to change the color dynamically, god
forbid if I have to apply a filter or animate a gradient. We do all sorts
of such things in our Flex apps today.
Vectors are one big advantage that Flash has over straight HTML apps. Now
that SVG is being considered more seriously in the HTML5 era, we have a
great chance to bring that goodness over. HTML5 apps that fit different
various screen dimensions is the holy grail that folks are trying to
achieve. Flex has a good chance to help in this area, because that is what
it specializes in.
Again as I said, both FXG and SVG have very good support for Bitmaps. So
there is no need to exclude this option at all. Those who want to use just
bitmaps for skinning can still go ahead and do that.
> >
> > Someone mentioned in the comments on a recent post that if we replaced
> > DisplayObject type with IDisplayObject we could use our own renderers. We
> > could then have in our Flex applications something like a
> > stageRenderer:IRenderer property / interface.
> >
> > Really, if we get this to work and apps run as silky smooth as games do
> > then it will put Flex / AIR on the map for apps.
> I have heard that the key to performance is the number of times you need to
> render your vectors to bitmaps. There are certain things (like zoom
> effects) that you don't think twice about using in vector rendering that
> kill you when trying to get to the GPU. I think I was told that if you can
> design your visuals as a set of fixed size visuals (which I guess games
> often do) then you get the benefits of GPU. Then the question is, if your
> visuals are fixed size, why not transcode them to bitmaps at compile time
> instead of run time?
>
My question to you is, how do you know during compile time that the visuals
are fixed size? Do you plan to do a static analysis of the code to see if
they are not being manipulated for dimensions, filters, gradients, etc.?
If it is a compiler switch, then yes, this approach would work. But IMHO
that would put an undue restriction on developers.
Bitmap spritesheets are great for games because the objects dont change in
size generally. But the kind of fluid layout apps Flex devs are used to
building, we still need vectors to do most work.
Maybe scale-9 skinning would fix some problems, but I am not sure if it
will solve everything.
I have also been reading some stuff about gpu accelerated SVG. I will do a
bit more research before posting links here.
Thanks,
Om
>
> --
> Alex Harui
> Flex SDK Team
> Adobe Systems, Inc.
> http://blogs.adobe.com/aharui
>
>
Re: Rendering layer in Flex <-> HTML5/JS solution
Posted by Harbs <ha...@gmail.com>.
Didn't they do away with GPU mode this past year?
I think you need to use Stage3D…
On Mar 20, 2013, at 6:15 PM, jude wrote:
> On Tue, Mar 19, 2013 at 8:08 PM, Om <bi...@gmail.com> wrote:
>>
>>
>> Please go on with the rambling :-) Its nice to know that others are
>> thinking about things I am thinking as well - and that I am crazy!
>>
>> BTW, I have a nice little carefully crafted XSLT that converts a basic FXG
>> node with just the Rect element to SVG. I am planning on tackling the
>> 'fill' element next. Once I have a decently working XSLT that simply
>> converts an FXG skin that can be directly rendered on a browser, I will
>> start posting my work for others to look at at hopefully, contribute.
>>
>
> Cool. :) I was reading a post yesterday about the massive performance
> improvements by rasterizing the vectors and then moving them to the GPU,
> http://goo.gl/WXkmA. I haven't seen this approach before and it may not
> have been possible before but with Stage 3d, Starling and WebGL it now is a
> choice. And if that works then we may be able to apply it to the Flex SDK
> and even HTML output (it renders at difference sizing so as you zoom in or
> out it maintains fidelity - I think).
>
> Someone mentioned in the comments on a recent post that if we replaced
> DisplayObject type with IDisplayObject we could use our own renderers. We
> could then have in our Flex applications something like a
> stageRenderer:IRenderer property / interface.
>
> Really, if we get this to work and apps run as silky smooth as games do
> then it will put Flex / AIR on the map for apps.
Re: Rendering layer in Flex <-> HTML5/JS solution
Posted by Alex Harui <ah...@adobe.com>.
On 3/20/13 9:55 AM, "jude" <fl...@gmail.com> wrote:
>
> Ha, yes. :) I was concerned that we may still need a drawing canvas at
> runtime or that we should keep looking into it. What if you change the
> color of a button? I was looking online and I think the HTML canvas has a
> draw command. So we could mix and match. We draw to the canvas and then
> rasterize it and load it into the dictionary replacing the last instance if
> it's been updated. If the canvas doesn't allow it we can also check webgl.
> I haven't had a chance to look at Flex JS but I will this weekend.
>
For sure, we will probably have to support runtime vector rasterization, but
we have to make it clear to folks that it could be very expensive. It might
be far cheaper to have two button bitmaps of different colors. Even a dozen
bitmaps might be less expensive.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: Rendering layer in Flex <-> HTML5/JS solution
Posted by jude <fl...@gmail.com>.
On Wed, Mar 20, 2013 at 11:38 AM, Alex Harui <ah...@adobe.com> wrote:
>
>
>
> On 3/20/13 9:15 AM, "jude" <fl...@gmail.com> wrote:
>
> > On Tue, Mar 19, 2013 at 8:08 PM, Om <bi...@gmail.com> wrote:
> >>
> >>
> >> Please go on with the rambling :-) Its nice to know that others are
> >> thinking about things I am thinking as well - and that I am crazy!
> >>
> >> BTW, I have a nice little carefully crafted XSLT that converts a basic
> FXG
> >> node with just the Rect element to SVG. I am planning on tackling the
> >> 'fill' element next. Once I have a decently working XSLT that simply
> >> converts an FXG skin that can be directly rendered on a browser, I will
> >> start posting my work for others to look at at hopefully, contribute.
> >>
> >
> > Cool. :) I was reading a post yesterday about the massive performance
> > improvements by rasterizing the vectors and then moving them to the GPU,
> > http://goo.gl/WXkmA. I haven't seen this approach before and it may not
> > have been possible before but with Stage 3d, Starling and WebGL it now
> is a
> > choice. And if that works then we may be able to apply it to the Flex SDK
> > and even HTML output (it renders at difference sizing so as you zoom in
> or
> > out it maintains fidelity - I think).
> Did I read that post correctly? I think it supports my choice of bitmaps
> over vectors for the future.
>
Ha, yes. :) I was concerned that we may still need a drawing canvas at
runtime or that we should keep looking into it. What if you change the
color of a button? I was looking online and I think the HTML canvas has a
draw command. So we could mix and match. We draw to the canvas and then
rasterize it and load it into the dictionary replacing the last instance if
it's been updated. If the canvas doesn't allow it we can also check webgl.
I haven't had a chance to look at Flex JS but I will this weekend.
>
> > Someone mentioned in the comments on a recent post that if we replaced
> > DisplayObject type with IDisplayObject we could use our own renderers. We
> > could then have in our Flex applications something like a
> > stageRenderer:IRenderer property / interface.
> >
> > Really, if we get this to work and apps run as silky smooth as games do
> > then it will put Flex / AIR on the map for apps.
> I have heard that the key to performance is the number of times you need to
> render your vectors to bitmaps. There are certain things (like zoom
> effects) that you don't think twice about using in vector rendering that
> kill you when trying to get to the GPU. I think I was told that if you can
> design your visuals as a set of fixed size visuals (which I guess games
> often do) then you get the benefits of GPU. Then the question is, if your
> visuals are fixed size, why not transcode them to bitmaps at compile time
> instead of run time?
>
Indeed although I can't confirm.
Harb, I think they said they were but I am still able to choose it as a
rendering mode in AIR 3.6. But in this context was using it interchangeably
with Stage3D for Flash and WebGL for browser.
Re: Rendering layer in Flex <-> HTML5/JS solution
Posted by Alex Harui <ah...@adobe.com>.
On 3/20/13 9:15 AM, "jude" <fl...@gmail.com> wrote:
> On Tue, Mar 19, 2013 at 8:08 PM, Om <bi...@gmail.com> wrote:
>>
>>
>> Please go on with the rambling :-) Its nice to know that others are
>> thinking about things I am thinking as well - and that I am crazy!
>>
>> BTW, I have a nice little carefully crafted XSLT that converts a basic FXG
>> node with just the Rect element to SVG. I am planning on tackling the
>> 'fill' element next. Once I have a decently working XSLT that simply
>> converts an FXG skin that can be directly rendered on a browser, I will
>> start posting my work for others to look at at hopefully, contribute.
>>
>
> Cool. :) I was reading a post yesterday about the massive performance
> improvements by rasterizing the vectors and then moving them to the GPU,
> http://goo.gl/WXkmA. I haven't seen this approach before and it may not
> have been possible before but with Stage 3d, Starling and WebGL it now is a
> choice. And if that works then we may be able to apply it to the Flex SDK
> and even HTML output (it renders at difference sizing so as you zoom in or
> out it maintains fidelity - I think).
Did I read that post correctly? I think it supports my choice of bitmaps
over vectors for the future.
>
> Someone mentioned in the comments on a recent post that if we replaced
> DisplayObject type with IDisplayObject we could use our own renderers. We
> could then have in our Flex applications something like a
> stageRenderer:IRenderer property / interface.
>
> Really, if we get this to work and apps run as silky smooth as games do
> then it will put Flex / AIR on the map for apps.
I have heard that the key to performance is the number of times you need to
render your vectors to bitmaps. There are certain things (like zoom
effects) that you don't think twice about using in vector rendering that
kill you when trying to get to the GPU. I think I was told that if you can
design your visuals as a set of fixed size visuals (which I guess games
often do) then you get the benefits of GPU. Then the question is, if your
visuals are fixed size, why not transcode them to bitmaps at compile time
instead of run time?
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: Rendering layer in Flex <-> HTML5/JS solution
Posted by jude <fl...@gmail.com>.
On Tue, Mar 19, 2013 at 8:08 PM, Om <bi...@gmail.com> wrote:
>
>
> Please go on with the rambling :-) Its nice to know that others are
> thinking about things I am thinking as well - and that I am crazy!
>
> BTW, I have a nice little carefully crafted XSLT that converts a basic FXG
> node with just the Rect element to SVG. I am planning on tackling the
> 'fill' element next. Once I have a decently working XSLT that simply
> converts an FXG skin that can be directly rendered on a browser, I will
> start posting my work for others to look at at hopefully, contribute.
>
Cool. :) I was reading a post yesterday about the massive performance
improvements by rasterizing the vectors and then moving them to the GPU,
http://goo.gl/WXkmA. I haven't seen this approach before and it may not
have been possible before but with Stage 3d, Starling and WebGL it now is a
choice. And if that works then we may be able to apply it to the Flex SDK
and even HTML output (it renders at difference sizing so as you zoom in or
out it maintains fidelity - I think).
Someone mentioned in the comments on a recent post that if we replaced
DisplayObject type with IDisplayObject we could use our own renderers. We
could then have in our Flex applications something like a
stageRenderer:IRenderer property / interface.
Really, if we get this to work and apps run as silky smooth as games do
then it will put Flex / AIR on the map for apps.