You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flex.apache.org by after24 <vi...@after24.net> on 2015/04/17 11:26:23 UTC
Flex Scroller optimization
Hello,
I have been working on the scroller component to improve the framerate of
the scrolling operations with an optimization based on the blitting
technique.
In this experiment, the Scroller component has a new boolean property named
cacheViewport.
When cacheViewPort is set to true, all scrolling operations are performed
according to the following cycle :
1 - A bitmap copy of the viewport is made just before the next scrolling
operation (large viewports are cached in multiple tiles if necessary)
2 - When the user starts a scrolling operation, the actual viewport is
replaced by its bitmap version and the scrolling is executed by drawing the
viewport according to its verticalScrollPosition and
horizontalScrollPosition properties.
3 - At the end of the scroll operation, the bitmap viewport is replaced by
its actual version.
4 - Between each scrolling operations, when the user interacts with the
viewport content, a process mark as 'dirty' every regions of the viewport
that are likely to have changed during those interactions. Every dirty
region is redrawn before the next scrolling operation.
Pros :
- 60 FPS scrolling, even on old devices.
- Performance is no longer dependant on the viewport complexity.
- Can be used with the list component if it contains a moderate number of
rows (useVirtualLayout must be set to false though)
Cons :
- There is a slight lag (dependant on the device processor) during the
viewport rasterization and the process of redrawing every dirty regions.
- Doesn't support virtualized layouts.
- Animated elements of the viewport freezes during scrolling operations.
- Render mode must be set to <renderMode>gpu</renderMode> on the current
version.
- Depending on its dimensions, the viewport rasterization process can
consume large amounts of memory.
It works well, even on old phones. I tested it on a nexus S, a nexus 4 and a
nexus 7 (however, I didn’t made tests on ios devices).
A demo application can be downloaded on the android play market here :
https://play.google.com/store/apps/details?id=air.fr.after24.ViewportCacheDemo
<https://play.google.com/store/apps/details?id=air.fr.after24.ViewportCacheDemo>
I opened a Jira ticket with the source code FLEX-34821
<https://issues.apache.org/jira/browse/FLEX-34821>
--
View this message in context: http://apache-flex-development.2333347.n4.nabble.com/Flex-Scroller-optimization-tp46074.html
Sent from the Apache Flex Development mailing list archive at Nabble.com.
Re:Re: Re:Re: Re:Flex Scroller optimization
Posted by DarkStone <da...@163.com>.
Hi after24,
I see the screen capture this time by using a proxy.
It seems the performance difference between both is significant.
Can you send me your Android demo (apk file) to my personal email? Which is DarkStone@163.com
I know you have a apk file on Google Play, but China blocks all google sites.
Thanks
DarkStone
2015-04-19
At 2015-04-19 01:03:48, "after24" <vi...@after24.net> wrote:
>DarkStone,Here is a Twitter link to the screen capture :
>https://twitter.com/after24_studio/status/589389289059942400
><https://twitter.com/after24_studio/status/589389289059942400> 1 ->Both
>tests are made under gpu render mode.2 ->The difference between the two
>techniques is significant, on a nexus 7 : - 25/30 fps with cacheAsBitmap
>- 60 fps with the blitting approch (and a frame budget around 8-10 ms)
>
>
>
>--
>View this message in context: http://apache-flex-development.2333347.n4.nabble.com/Flex-Scroller-optimization-tp46074p46084.html
>Sent from the Apache Flex Development mailing list archive at Nabble.com.
Re: Re:Re: Re:Re: Re:Re: Re:Flex Scroller optimization
Posted by after24 <vi...@after24.net>.
Hello Darkstone,
Thanks for those tests. It would be very interesting that other users share
the results they get on other devices, especially on iOS.
I am deeply convinced that UI smoothness is a key parameter of the user
experience and has a large impact on the perception of the quality of an
application : Google made this statement 3 years ago and responded to the
choppiness of its OS with « project butter » introduced in android Jelly
Bean. I am also profoundly convinced that the lack of smoothness of Flex
mobile (on a large majority of devices) prevents many developers from using
it and is an argument for its detractors. Feather UI is a solution, it works
perfectly well and is evolving toward something more friendly for Flex
developers with the future integration of mxml, but it’s not the swiss knife
that flex is… especially for developers who build Flex based RIA with the
need of producing a mobile version of their product.
I digress a little from the initial subject but we do not say enough how
Flex is a fantastic tool. It’s clear that HTML5 is catching up and is well
suited for a lot of things but I am not in a hurry to jump in the .js train
because it seem’s to me that the existing tools are not stable, not in the
sense of execution stability but rather in evolution stability : the joke of
« a new .js framework every day » illustrates this constant evolution state
of a technology that is not quite ready for the moment when it comes to
product large scale applications in optimal conditions. It’s easy to be
scared when you follow trends on twitter and feeling like a dinosaur because
you continue to use a technology that is less popular than others. I am not
against learning a new technology, not at all, but it must be for a good
reason and not because the hype is on a particular tool for more than 2
weeks. My job (and I think the job of a lot of people on this list) is not
to build dev tools or master 10 langages and 30 frameworks : my job is to
build applications to address clients needs, and right now, Flex is the
perfect solution. My clients are happy and don’t care about the need of
having flash player installed on their computer. Brainless flash haters or
allegedly journalists desperate to kill something, because buzz words like
kill or dead fills the void of knowledge and objectivity in journalism 5.0,
are far less numerous than happy clients and users who don’t care about the
engine brand if the engine is powerful… and smooth.
This is the reason why I try to optimize flex mobile UI animation. The
cacheViewport experiment is not a production ready feature but it clearly
shows that there is space for improvements with the current SDK. Modern
devices are more suited to run flex mobile (Darkstone, I am very impressed
with the results on your device without using the cacheViewport
optimization) but how many years will it take before every user device can
run any flex mobile project at 60 FPS ?
Regarding this experimentation, it’s right that using this technique in the
viewport itself rather than in the scroller using the touchInteraction
events is definetely not the safer solution : I’m pretty sure that mustella
won’t be very happy with the changes I made on the Scroller code even if I
tried to make them as light as possible. But like Jude says, having an out
of the box « magic button » will be a nice feature for every Flex user. It
took me a lot of time to find the most performant blitting approach and to
integrate it to the scroller, and during this process I found out that
mobile processors are not the only ones responsible for jerky animations :
the kinetic of the spark.effects.TrowEffect is far from being perfect (I
will try to use the Greensock throwPropsPlugin to see if it’s make a
significant difference).
Darkstone,
« for old smart mobile devices, which have low performance CPUs and GPUs,
and small RAMs, your cacheViewport solution is very good for use in low
performance CPUs, but the downside is your solution requires a lot memory
consumption, which is quite expensive to bear for those old devices »
Concerning this downside, it’s true that caching the whole viewport can
consume a considerable amount of memory but it should be balanced by the
fact that low end devices usually have a low pixel density screen.
Nevertheless this technique is definitely not appropriate for cases like
lists with a very large number of rows where the contentHeight property can
reach dozen of thousands of pixels, not to mention the fact that its
initialization will take a very long time because each row will lead to the
creation of an item renderer since the layout can't be virtualized.
« it seems you don't clear your bitmap cache of the viewport, which I think
you should put it into consideration »
this is deliberate because the rasterization of the viewport is a costly
process (200-300 ms on my device for the content of the demo) and should be
done only when necessary, i.e. right before the first scrolling operation or
each times the size of the viewport or its content changed. A second process
marks as « dirty » every regions of the viewport that are likely to have
changed during user interactions and every dirty region is redrawn before
the next scrolling operation.
I have a problem with this process which is longer than expected even when a
small region is redrawn :
- The bitmapData.draw call that update the dirty region of the viewport
cache is relatively fast (10-25 ms) but it seems to trigger a long phase of
displayList rendering that takes up to 200 ms and causes a lag effect during
a scroll operation that follows a modification in the UI (for example a
switch that is toggled by the user). I don't understand why, so if someone
has an idea about that.
So to get back to the question of the bitmap cache memory management, each
bitmapData tile is not disposed until the next caching operation. I planned
to added listeners on the viewport to also call the disposeTiles() method
when the viewport is removed from the stage.
If someone wants to take a look at the code, it’s very simple :
- The spark.components.supportClasses.ViewportCache class implements the
IViewport interface and composed two other classes :
- The spark.components.supportClasses.ViewportCacheRasterizer class in
charge of the rasterization of the viewport and the redrawing of the dirty
regions
- The spark.components.supportClasses.ViewportCacheDirtRegions class that
watches any changes in the viewport and established a map of every dirty
region that need to be redrawn
A viewPortCache(Boolean) property is added to the scroller class with two
additional methods : installViewportCache() and uninstallViewportCache()
that are called respectively at the beginning and the end of a scroll
operation.
Sorry for the long post and the digression :-)
--
View this message in context: http://apache-flex-development.2333347.n4.nabble.com/Flex-Scroller-optimization-tp46074p46116.html
Sent from the Apache Flex Development mailing list archive at Nabble.com.
Re:Re: Re:Re: Re:Re: Re:Flex Scroller optimization
Posted by DarkStone <da...@163.com>.
Hi after24,
I installed your apk on my Android phone, here is the detail:
My Android phone hardware specification:
Model: M1 Note
5.5 inch screen, 1920x1080, 401 ppi
2GB RAM, 16GB ROM
ARM® Cortex®-A53 1.7GHz x 8 (64 bit)
Mali T760 MP2/700MHz GPU
Full specification go here:
http://www.meizu.com/en/products/m1note/spec.html
By the way, the price of this phone is only $160 US dollars, yeah it's cheap and powerful : D
Now here is the Scout data of your apk:
Your apk file:
https://issues.apache.org/jira/secure/attachment/12726429/ViewportCacheDemo.apk
Scout data file:
https://issues.apache.org/jira/secure/attachment/12726495/ViewportCacheDemo.flm
Swiping the list like madly with cacheViewport set to false:
Average framerate: 59.3 fps
Average CPU usage: 73%
Peak CPU usage: 100%
Total Memory usage: 28 MB
https://issues.apache.org/jira/secure/attachment/12726496/01%20Swiping%20the%20list%20like%20madly%20with%20cacheViewport%20set%20to%20false.png
Swiping the list like madly with cacheViewport set to true:
Average framerate: 58.9 fps
Average CPU usage: 33%
Peak CPU usage: 105%
Total Memory usage: 82 MB
https://issues.apache.org/jira/secure/attachment/12726497/02%20Swiping%20the%20list%20like%20madly%20with%20cacheViewport%20set%20to%20true.png
With cacheViewport set to true, the CPU usage is pretty low, which is very good, espeically for old devices, but the Total Memory usage is a little bit too high, and after the scrolling has stopped, it seems you don't clear your bitmap cache of the viewport, which I think you should put it into consideration.
With the testing result, here is an interesting topic to discuss:
The modern mainstream smart mobile devices, which have cheap prices, high performance CPUs and GPUs, as you can see, they can already handle the Flex scrolling pretty well.
For old smart mobile devices, which have low performance CPUs and GPUs, and small RAMs, your cacheViewport solution is very good for use in low performance CPUs, but the downside is your solution requires a lot memory consumption, which is quite expensive to bear for those old devices.
I would like to see a more complex list, with a lot of pictures, texts, controls mixed in it, and then I'd like to test it again, to see if cacheViewport = true has more obvious advantages over cacheViewport = false.
All in all, I think your work is worthy, although it still needs more scenarios to test and more people to share their thoughts on.
Good work!
DarkStone
2015-04-20
At 2015-04-19 16:55:32, "after24" <vi...@after24.net> wrote:
>Hello DarkStone,
>
>Can you access the apache flex JIRA ? If you can, I could attach the .apk of
>the demo version.
>
>
>
>--
>View this message in context: http://apache-flex-development.2333347.n4.nabble.com/Flex-Scroller-optimization-tp46074p46090.html
>Sent from the Apache Flex Development mailing list archive at Nabble.com.
Re: Re:Re: Re:Re: Re:Flex Scroller optimization
Posted by after24 <vi...@after24.net>.
Hello DarkStone,
Can you access the apache flex JIRA ? If you can, I could attach the .apk of
the demo version.
--
View this message in context: http://apache-flex-development.2333347.n4.nabble.com/Flex-Scroller-optimization-tp46074p46090.html
Sent from the Apache Flex Development mailing list archive at Nabble.com.
Re:Re: Re:Re: Re:Flex Scroller optimization
Posted by DarkStone <da...@163.com>.
Hi jude,
I agree what you said, I just need more tests to make sure : )
I think technically it is not required to modify the Scroller.as to do this, it can be done by writting a subclass of the Group, listens for its touchInteractionStart and touchInteractionEnd events, and override its horizontalScrollPosition and verticalScrollPosition properties to apply the blitting technique.
By the way, if after24 sees this, please consider this approach I described above, because essentially the Scroller just listens for its viewport's touch interaction events and update its viewport's horizontalScrollPosition and verticalScrollPosition, so the Scroller is not the key to the blitting technique, these works can all be done in a subclass of the Group.
DarkStone
2015-04-19
At 2015-04-19 08:43:57, "jude" <fl...@gmail.com> wrote:
>Darkstone, I like that you can do everything now with the current scroller
>but that requires you to know what to do. The advantage of this is that you
>only have to set a property on the scroller to get the benefits. If it is
>able to be turned off then it can be added to the class without breaking
>anything and people can start using it in their applications. Then you
>don't have to use it but if you want to you can set the cacheViewport
>property where it makes sense. +1
>
>On Sat, Apr 18, 2015 at 10:03 AM, after24 <vi...@after24.net> wrote:
>
>> DarkStone,Here is a Twitter link to the screen capture :
>> https://twitter.com/after24_studio/status/589389289059942400
>> <https://twitter.com/after24_studio/status/589389289059942400> 1 ->Both
>> tests are made under gpu render mode.2 ->The difference between the two
>> techniques is significant, on a nexus 7 : - 25/30 fps with
>> cacheAsBitmap
>> - 60 fps with the blitting approch (and a frame budget around 8-10 ms)
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-flex-development.2333347.n4.nabble.com/Flex-Scroller-optimization-tp46074p46084.html
>> Sent from the Apache Flex Development mailing list archive at Nabble.com.
>>
Re: Re:Re: Re:Flex Scroller optimization
Posted by jude <fl...@gmail.com>.
Darkstone, I like that you can do everything now with the current scroller
but that requires you to know what to do. The advantage of this is that you
only have to set a property on the scroller to get the benefits. If it is
able to be turned off then it can be added to the class without breaking
anything and people can start using it in their applications. Then you
don't have to use it but if you want to you can set the cacheViewport
property where it makes sense. +1
On Sat, Apr 18, 2015 at 10:03 AM, after24 <vi...@after24.net> wrote:
> DarkStone,Here is a Twitter link to the screen capture :
> https://twitter.com/after24_studio/status/589389289059942400
> <https://twitter.com/after24_studio/status/589389289059942400> 1 ->Both
> tests are made under gpu render mode.2 ->The difference between the two
> techniques is significant, on a nexus 7 : - 25/30 fps with
> cacheAsBitmap
> - 60 fps with the blitting approch (and a frame budget around 8-10 ms)
>
>
>
> --
> View this message in context:
> http://apache-flex-development.2333347.n4.nabble.com/Flex-Scroller-optimization-tp46074p46084.html
> Sent from the Apache Flex Development mailing list archive at Nabble.com.
>
Re: Re:Re: Re:Flex Scroller optimization
Posted by after24 <vi...@after24.net>.
DarkStone,Here is a Twitter link to the screen capture :
https://twitter.com/after24_studio/status/589389289059942400
<https://twitter.com/after24_studio/status/589389289059942400> 1 ->Both
tests are made under gpu render mode.2 ->The difference between the two
techniques is significant, on a nexus 7 : - 25/30 fps with cacheAsBitmap
- 60 fps with the blitting approch (and a frame budget around 8-10 ms)
--
View this message in context: http://apache-flex-development.2333347.n4.nabble.com/Flex-Scroller-optimization-tp46074p46084.html
Sent from the Apache Flex Development mailing list archive at Nabble.com.
Re:Re: Re:Flex Scroller optimization
Posted by DarkStone <da...@163.com>.
Hi after24,
I cannot open the link you provided (maybe it's because I'm in China):
https://www.imageupload.co.uk/image/ZTUi
So, I just wanna ask you 2 questions:
1: Are the both tests using the "gpu" for renderMode? (If not, please do so)
2: The performance gap between the two techniques, are they close to each other, or have significant difference?
(In my tests, they are very close)
Thanks.
DarkStone
2015-04-18
At 2015-04-18 16:48:37, "after24" <vi...@after24.net> wrote:
>Hello Darkstone,
>
>I made a comparison (on a nexus 7) using your approch with the same viewport
>content as in my demo application.
>
>Here is a link to a scout screen capture of the two methods that shows the
>performance gap between the two techniques :
>https://www.imageupload.co.uk/image/ZTUi
><https://www.imageupload.co.uk/image/ZTUi>
>
>
>
>
>--
>View this message in context: http://apache-flex-development.2333347.n4.nabble.com/Flex-Scroller-optimization-tp46074p46081.html
>Sent from the Apache Flex Development mailing list archive at Nabble.com.
Re: Re:Flex Scroller optimization
Posted by after24 <vi...@after24.net>.
Hello Darkstone,
I made a comparison (on a nexus 7) using your approch with the same viewport
content as in my demo application.
Here is a link to a scout screen capture of the two methods that shows the
performance gap between the two techniques :
https://www.imageupload.co.uk/image/ZTUi
<https://www.imageupload.co.uk/image/ZTUi>
--
View this message in context: http://apache-flex-development.2333347.n4.nabble.com/Flex-Scroller-optimization-tp46074p46081.html
Sent from the Apache Flex Development mailing list archive at Nabble.com.
Re:Flex Scroller optimization
Posted by DarkStone <da...@163.com>.
Hi after24,
I believe I had replied you on this topic, in case you missed it, here is the link (at the 13th floor):
http://apache-flex-development.2333347.n4.nabble.com/Scroller-optimization-td41774.html#a42858
What I'am saying is, by simply using the cacheAsBitmap mechanism, you can achieve the same goal, so there is no need to modify the Scroller.as source code.
Here is the sample code for how to use cacheAsBitmap to improve the scrolling performance:
<s:Scroller width="100%" height="100%">
<s:Group>
<s:DataGroup id="realContent" touchInteractionStart="realContent.mouseChildren=false; realContent.cacheAsBitmap=true;" touchInteractionEnd="realContent.mouseChildren=true; realContent.cacheAsBitmap=false;" itemRenderer="com.test.renderers.MyItemRenderer" dataProvider="{someData}" width="100%">
<s:layout>
<s:VerticalLayout useVirtualLayout="false"/>
</s:layout>
</s:DataGroup>
</s:Group>
</s:Scroller>
The essential elements are:
1. You have to put a Group inside the Scroller, then put the real content (<s:DataGroup id="realContent">) inside that Group.
2. The layout of the realContent, you have to set its useVirtualLayout property to false (the default value is false).
3. Listen for the realContent's touchInteractionStart and touchInteractionEnd event:
When touchInteractionStart, disable any mouse and touch interactions of the realContent's mouseChildren by setting realContent.mouseChildren = false, then turn on the cache for realContent by setting realContent.cacheAsBitmap = true.
When touchInteractionEnd, reverse the changes you made for the realContent.
That's it! It's very easy to improve the scrolling performance by doing so. It's the same scrolling performance result when compared to the BitmapData.copyPixels() approach, yet cacheAsBitmap is much easier to understand and handle.
If you do not believe me, please test it yourself : )
DarkStone
2015-04-18
At 2015-04-17 17:26:23, "after24" <vi...@after24.net> wrote:
>Hello,
>
>I have been working on the scroller component to improve the framerate of
>the scrolling operations with an optimization based on the blitting
>technique.
>In this experiment, the Scroller component has a new boolean property named
>cacheViewport.
>
>When cacheViewPort is set to true, all scrolling operations are performed
>according to the following cycle :
>
>1 - A bitmap copy of the viewport is made just before the next scrolling
>operation (large viewports are cached in multiple tiles if necessary)
>
>2 - When the user starts a scrolling operation, the actual viewport is
>replaced by its bitmap version and the scrolling is executed by drawing the
>viewport according to its verticalScrollPosition and
>horizontalScrollPosition properties.
>
>3 - At the end of the scroll operation, the bitmap viewport is replaced by
>its actual version.
>
>4 - Between each scrolling operations, when the user interacts with the
>viewport content, a process mark as 'dirty' every regions of the viewport
>that are likely to have changed during those interactions. Every dirty
>region is redrawn before the next scrolling operation.
>
>
>Pros :
>
>- 60 FPS scrolling, even on old devices.
>- Performance is no longer dependant on the viewport complexity.
>- Can be used with the list component if it contains a moderate number of
>rows (useVirtualLayout must be set to false though)
>
>Cons :
>
>- There is a slight lag (dependant on the device processor) during the
>viewport rasterization and the process of redrawing every dirty regions.
>- Doesn't support virtualized layouts.
>- Animated elements of the viewport freezes during scrolling operations.
>- Render mode must be set to <renderMode>gpu</renderMode> on the current
>version.
>- Depending on its dimensions, the viewport rasterization process can
>consume large amounts of memory.
>
>
>It works well, even on old phones. I tested it on a nexus S, a nexus 4 and a
>nexus 7 (however, I didn’t made tests on ios devices).
>
>A demo application can be downloaded on the android play market here :
>https://play.google.com/store/apps/details?id=air.fr.after24.ViewportCacheDemo
><https://play.google.com/store/apps/details?id=air.fr.after24.ViewportCacheDemo>
>
>I opened a Jira ticket with the source code FLEX-34821
><https://issues.apache.org/jira/browse/FLEX-34821>
>
>
>
>--
>View this message in context: http://apache-flex-development.2333347.n4.nabble.com/Flex-Scroller-optimization-tp46074.html
>Sent from the Apache Flex Development mailing list archive at Nabble.com.