You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@flex.apache.org by "christofer.dutz@c-ware.de" <ch...@c-ware.de> on 2013/10/16 10:26:40 UTC

Sorting out Memory issues

Hi,

I am currently having Major Memory issues, which I haven't quite managed to sort out.

In my application (Apache Flex 4.10 ... but wasn't any different in older and/or Adobe Versions) I am loading a list of objects and this list is updated via server-push (Streaming AMF Connection). For testing, I increased the amount of data sent back to the Client when loading the list. Currently about 12000 Objects are returned using about 107MB (according to Adobe Scout).

The returned list of objects is displayed in a Flexicious Ultimate DataGrid (The Performance of this is very sattisfying considering the amount of data it has to handle).

There is a Feature to force a reload of the list. As removing all elements of an ArrayList and then adding 12000 Objects caused Major Performance problems, I changed the code to replace the source array of the ArrayCollection and then to make the list update itself after that. Each time the list is reloaded 107MB are used but I can't see any Memory being freed.

Is there something wrong with my Approach? Do I have to Keep something else in mind? Are there some hidden internals keeping a reference to my objects so they aren't cleaned up?

Help greatly appreciated.

Chris

Re: AW: Sorting out Memory issues

Posted by Alex Harui <ah...@adobe.com>.


On 10/17/13 2:11 AM, "christofer.dutz@c-ware.de"
<ch...@c-ware.de> wrote:

>Hi Marcus,
>
>yesterday I did a Little more investigating. I could find out that there
>was an insane amount of Events generated the UUIDs of These seem to have
>consumed about 60MB. The reason for this seems to be that I was using the
>Default GraniteDS Templates (With minor modifications to make the model
>classes work with BlazeDS) in my flexmojos build to generate ActionScript
>model classes from my Java classes. This build generated the model
>classes with [Bindable] on class Level. I modified my templates to simply
>create non-bindable classes and the Event-Inferno seems to have been
>resolved.
Yes, there is a price to pay to get change events from the properties of a
data class.  If you don't need them, then don't use [Bindable].

>
>But I can confirm that the UUID Generation seems to cause a significant
>amount of CPU and Memory usage. Eventually it would be good to address
>this on a Framework Level.
You can have your data classes implement mx.core.IUID and that should
bypass UUID generation.  In theory, most folks have a unique identifier in
their data items already (customer account #, employee id, etc).

>
>While investigating the "source" method of ArrayCollection (I could have
>beet that this was implemented greatly different in Flex 3.x), I could
>see that internally the Array is wrapped by an ArrayList and that there
>is an alternate implementation with the same Name form an Apache Flex
>package which seems to work with vectors. What's actually the difference
>between the two ArrayList implementations? Could it be beneficial to
>Switch to the new Apache implementation officially in the ArrayCollection
>class?
If you have written your code to work with data sets as Vectors, then not
having to switch back to Array/ArrayList is helpful and can assist in some
type-checking.

However, I believe there are tricks to serializing/deserializing, and
there may be performance issues when casting places where the API loses
the fact that it is a Vector like in a List's selectedItem.  So the
benefits may vary.

-Alex

AW: Sorting out Memory issues

Posted by "christofer.dutz@c-ware.de" <ch...@c-ware.de>.

Hi Marcus,

yesterday I did a Little more investigating. I could find out that there was an insane amount of Events generated the UUIDs of These seem to have consumed about 60MB. The reason for this seems to be that I was using the Default GraniteDS Templates (With minor modifications to make the model classes work with BlazeDS) in my flexmojos build to generate ActionScript model classes from my Java classes. This build generated the model classes with [Bindable] on class Level. I modified my templates to simply create non-bindable classes and the Event-Inferno seems to have been resolved.

But I can confirm that the UUID Generation seems to cause a significant amount of CPU and Memory usage. Eventually it would be good to address this on a Framework Level.

While investigating the "source" method of ArrayCollection (I could have beet that this was implemented greatly different in Flex 3.x), I could see that internally the Array is wrapped by an ArrayList and that there is an alternate implementation with the same Name form an Apache Flex package which seems to work with vectors. What's actually the difference between the two ArrayList implementations? Could it be beneficial to Switch to the new Apache implementation officially in the ArrayCollection class?

Chris

________________________________________
Von: Marcus Wilkinson [wilkinsonmarcus@gmail.com]
Gesendet: Mittwoch, 16. Oktober 2013 12:00
An: users@flex.apache.org
Betreff: Re: Sorting out Memory issues

Chris,

This may be unrelated, but we were having big performance problems during
deserialisation of many separate ArrayCollections. The biggest bottleneck
was in the constructor of ArrayCollection, which would use an ArrayList in
its backing collection. Inside that function there is code:
    public function ArrayList(source:Array = null)
    {
        super();
       ...
        _uid = UIDUtil.createUID();
    }

The UIDUtil.createUID was taking way too long and some of the memory didn't
seem to get freed in a timely manner.
We monkey patched ArrayList to return a quicker UID:

private static const ARRAY_LIST_UID_PREFIX:String = "_array_list_uid_";
 private static var _count:int = Number.MIN_VALUE;//int.MIN_VALUE;
 private static function getUID():String {
 return ARRAY_LIST_UID_PREFIX+String(_count++)+String(getTimer());
}

Probably not the cleanest way to do this but it runs much more quickly.
Thought it may help you or others in this situation. I haven't raised an
apache flex bug because it seemed pretty specific to our situation.

Marcus

On 16 October 2013 09:26, christofer.dutz@c-ware.de <
christofer.dutz@c-ware.de> wrote:

> Hi,
>
> I am currently having Major Memory issues, which I haven't quite managed
> to sort out.
>
> In my application (Apache Flex 4.10 ... but wasn't any different in older
> and/or Adobe Versions) I am loading a list of objects and this list is
> updated via server-push (Streaming AMF Connection). For testing, I
> increased the amount of data sent back to the Client when loading the list.
> Currently about 12000 Objects are returned using about 107MB (according to
> Adobe Scout).
>
> The returned list of objects is displayed in a Flexicious Ultimate
> DataGrid (The Performance of this is very sattisfying considering the
> amount of data it has to handle).
>
> There is a Feature to force a reload of the list. As removing all elements
> of an ArrayList and then adding 12000 Objects caused Major Performance
> problems, I changed the code to replace the source array of the
> ArrayCollection and then to make the list update itself after that. Each
> time the list is reloaded 107MB are used but I can't see any Memory being
> freed.
>
> Is there something wrong with my Approach? Do I have to Keep something
> else in mind? Are there some hidden internals keeping a reference to my
> objects so they aren't cleaned up?
>
> Help greatly appreciated.
>
> Chris
>

AW: Sorting out Memory issues

Posted by "christofer.dutz@c-ware.de" <ch...@c-ware.de>.

Hi Marcus,

in another Project I remember us Setting BlazeDS to send legacy-collections which resulted in Arrays being sent. This reduced the Memory footprint and the serialization Speed dramatically. But I think this doesn't help in this case as it seems to be the case that the Problem seems to be something else still having a reference to my objects in the main list.

Chris

________________________________________
Von: Marcus Wilkinson [wilkinsonmarcus@gmail.com]
Gesendet: Mittwoch, 16. Oktober 2013 12:00
An: users@flex.apache.org
Betreff: Re: Sorting out Memory issues

Chris,

This may be unrelated, but we were having big performance problems during
deserialisation of many separate ArrayCollections. The biggest bottleneck
was in the constructor of ArrayCollection, which would use an ArrayList in
its backing collection. Inside that function there is code:
    public function ArrayList(source:Array = null)
    {
        super();
       ...
        _uid = UIDUtil.createUID();
    }

The UIDUtil.createUID was taking way too long and some of the memory didn't
seem to get freed in a timely manner.
We monkey patched ArrayList to return a quicker UID:

private static const ARRAY_LIST_UID_PREFIX:String = "_array_list_uid_";
 private static var _count:int = Number.MIN_VALUE;//int.MIN_VALUE;
 private static function getUID():String {
 return ARRAY_LIST_UID_PREFIX+String(_count++)+String(getTimer());
}

Probably not the cleanest way to do this but it runs much more quickly.
Thought it may help you or others in this situation. I haven't raised an
apache flex bug because it seemed pretty specific to our situation.

Marcus

On 16 October 2013 09:26, christofer.dutz@c-ware.de <
christofer.dutz@c-ware.de> wrote:

> Hi,
>
> I am currently having Major Memory issues, which I haven't quite managed
> to sort out.
>
> In my application (Apache Flex 4.10 ... but wasn't any different in older
> and/or Adobe Versions) I am loading a list of objects and this list is
> updated via server-push (Streaming AMF Connection). For testing, I
> increased the amount of data sent back to the Client when loading the list.
> Currently about 12000 Objects are returned using about 107MB (according to
> Adobe Scout).
>
> The returned list of objects is displayed in a Flexicious Ultimate
> DataGrid (The Performance of this is very sattisfying considering the
> amount of data it has to handle).
>
> There is a Feature to force a reload of the list. As removing all elements
> of an ArrayList and then adding 12000 Objects caused Major Performance
> problems, I changed the code to replace the source array of the
> ArrayCollection and then to make the list update itself after that. Each
> time the list is reloaded 107MB are used but I can't see any Memory being
> freed.
>
> Is there something wrong with my Approach? Do I have to Keep something
> else in mind? Are there some hidden internals keeping a reference to my
> objects so they aren't cleaned up?
>
> Help greatly appreciated.
>
> Chris
>

Re: Sorting out Memory issues

Posted by Marcus Wilkinson <wi...@gmail.com>.

Chris,

This may be unrelated, but we were having big performance problems during
deserialisation of many separate ArrayCollections. The biggest bottleneck
was in the constructor of ArrayCollection, which would use an ArrayList in
its backing collection. Inside that function there is code:
    public function ArrayList(source:Array = null)
    {
        super();
       ...
        _uid = UIDUtil.createUID();
    }

The UIDUtil.createUID was taking way too long and some of the memory didn't
seem to get freed in a timely manner.
We monkey patched ArrayList to return a quicker UID:

private static const ARRAY_LIST_UID_PREFIX:String = "_array_list_uid_";
 private static var _count:int = Number.MIN_VALUE;//int.MIN_VALUE;
 private static function getUID():String {
 return ARRAY_LIST_UID_PREFIX+String(_count++)+String(getTimer());
}

Probably not the cleanest way to do this but it runs much more quickly.
Thought it may help you or others in this situation. I haven't raised an
apache flex bug because it seemed pretty specific to our situation.

Marcus


On 16 October 2013 09:26, christofer.dutz@c-ware.de <
christofer.dutz@c-ware.de> wrote:

> Hi,
>
> I am currently having Major Memory issues, which I haven't quite managed
> to sort out.
>
> In my application (Apache Flex 4.10 ... but wasn't any different in older
> and/or Adobe Versions) I am loading a list of objects and this list is
> updated via server-push (Streaming AMF Connection). For testing, I
> increased the amount of data sent back to the Client when loading the list.
> Currently about 12000 Objects are returned using about 107MB (according to
> Adobe Scout).
>
> The returned list of objects is displayed in a Flexicious Ultimate
> DataGrid (The Performance of this is very sattisfying considering the
> amount of data it has to handle).
>
> There is a Feature to force a reload of the list. As removing all elements
> of an ArrayList and then adding 12000 Objects caused Major Performance
> problems, I changed the code to replace the source array of the
> ArrayCollection and then to make the list update itself after that. Each
> time the list is reloaded 107MB are used but I can't see any Memory being
> freed.
>
> Is there something wrong with my Approach? Do I have to Keep something
> else in mind? Are there some hidden internals keeping a reference to my
> objects so they aren't cleaned up?
>
> Help greatly appreciated.
>
> Chris
>

RE: Sorting out Memory issues

Posted by Maurice Amsellem <ma...@systar.com>.

I don't know what is your level of knowledge, so I apologize if saying something obvious to you.

Keep in mind that when you assign the source property of a list, it will trigger a CollectionEvent (with kind= RESET) that the Flexicious DataGrid (or s:DataGrid, s:List) will catch to do whatever.

Also, ArrayList has a mechanism for tracking updates to individual items ( startTrackUpdate / stopTrackUpdate ) that will set a listener on each item in your list, if it's an IEventDispatcher.
(see ArrayList set source ).
I don't think this is the cause.

Maurice

-----Message d'origine-----
De : christofer.dutz@c-ware.de [mailto:christofer.dutz@c-ware.de]
Envoyé : mercredi 16 octobre 2013 12:25
À : users@flex.apache.org
Objet : AW: Sorting out Memory issues

Hi Maurice,

Hmmm ... I think I could try that. Think it would be even easier to comment out the list entirely in a first step as reloading and storing of data is handled Independent of the displaying component. So' I'd be loading of data and refreshing of the list and see if everything is cleaned up correctlry. If that doesn't have the same Memory issues looking as Spark Grids woud be good.

Chris

________________________________________
Von: Maurice Amsellem [maurice.amsellem@systar.com]
Gesendet: Mittwoch, 16. Oktober 2013 12:16
An: users@flex.apache.org
Betreff: RE: Sorting out Memory issues

Can you try doing the same with a spark DataGrid, or even a simple spark List, to see if previous items are locked by the ArrayList (which I doubt) or by the Flexicious DataGrid (maybe caching, or adding events, or something).

Maurice

-----Message d'origine-----
De : christofer.dutz@c-ware.de [mailto:christofer.dutz@c-ware.de]
Envoyé : mercredi 16 octobre 2013 10:27
À : users@flex.apache.org
Objet : Sorting out Memory issues

Hi,

I am currently having Major Memory issues, which I haven't quite managed to sort out.

In my application (Apache Flex 4.10 ... but wasn't any different in older and/or Adobe Versions) I am loading a list of objects and this list is updated via server-push (Streaming AMF Connection). For testing, I increased the amount of data sent back to the Client when loading the list. Currently about 12000 Objects are returned using about 107MB (according to Adobe Scout).

The returned list of objects is displayed in a Flexicious Ultimate DataGrid (The Performance of this is very sattisfying considering the amount of data it has to handle).

There is a Feature to force a reload of the list. As removing all elements of an ArrayList and then adding 12000 Objects caused Major Performance problems, I changed the code to replace the source array of the ArrayCollection and then to make the list update itself after that. Each time the list is reloaded 107MB are used but I can't see any Memory being freed.

Is there something wrong with my Approach? Do I have to Keep something else in mind? Are there some hidden internals keeping a reference to my objects so they aren't cleaned up?

Help greatly appreciated.

Chris

RE: Sorting out Memory issues

Posted by Maurice Amsellem <ma...@systar.com>.

Flexicious DataGrid is also excellent :-)

-----Message d'origine-----
De : christofer.dutz@c-ware.de [mailto:christofer.dutz@c-ware.de]
Envoyé : mercredi 16 octobre 2013 12:25
À : users@flex.apache.org
Objet : AW: Sorting out Memory issues

Hi Maurice,

Chris

Maurice

-----Message d'origine-----
De : christofer.dutz@c-ware.de [mailto:christofer.dutz@c-ware.de]
Envoyé : mercredi 16 octobre 2013 10:27
À : users@flex.apache.org
Objet : Sorting out Memory issues

Hi,

I am currently having Major Memory issues, which I haven't quite managed to sort out.

The returned list of objects is displayed in a Flexicious Ultimate DataGrid (The Performance of this is very sattisfying considering the amount of data it has to handle).

Is there something wrong with my Approach? Do I have to Keep something else in mind? Are there some hidden internals keeping a reference to my objects so they aren't cleaned up?

Help greatly appreciated.

Chris

AW: Sorting out Memory issues

Posted by "christofer.dutz@c-ware.de" <ch...@c-ware.de>.

Hi Maurice,

Chris

Maurice

-----Message d'origine-----
De : christofer.dutz@c-ware.de [mailto:christofer.dutz@c-ware.de]
Envoyé : mercredi 16 octobre 2013 10:27
À : users@flex.apache.org
Objet : Sorting out Memory issues

Hi,

I am currently having Major Memory issues, which I haven't quite managed to sort out.

The returned list of objects is displayed in a Flexicious Ultimate DataGrid (The Performance of this is very sattisfying considering the amount of data it has to handle).

Is there something wrong with my Approach? Do I have to Keep something else in mind? Are there some hidden internals keeping a reference to my objects so they aren't cleaned up?

Help greatly appreciated.

Chris

RE: Sorting out Memory issues

Posted by Maurice Amsellem <ma...@systar.com>.

Maurice

-----Message d'origine-----
De : christofer.dutz@c-ware.de [mailto:christofer.dutz@c-ware.de]
Envoyé : mercredi 16 octobre 2013 10:27
À : users@flex.apache.org
Objet : Sorting out Memory issues

Hi,

I am currently having Major Memory issues, which I haven't quite managed to sort out.

The returned list of objects is displayed in a Flexicious Ultimate DataGrid (The Performance of this is very sattisfying considering the amount of data it has to handle).

Is there something wrong with my Approach? Do I have to Keep something else in mind? Are there some hidden internals keeping a reference to my objects so they aren't cleaned up?

Help greatly appreciated.

Chris

Re: Sorting out Memory issues

Posted by Alex Harui <ah...@adobe.com>.

I still haven't used Scout, but the Flash Builder Profiler has helped me
track down many issues like this.  It should show you what is referencing
those old objects.

On 10/16/13 1:26 AM, "christofer.dutz@c-ware.de"
<ch...@c-ware.de> wrote:

>Hi,
>
>I am currently having Major Memory issues, which I haven't quite managed
>to sort out.
>
>In my application (Apache Flex 4.10 ... but wasn't any different in older
>and/or Adobe Versions) I am loading a list of objects and this list is
>updated via server-push (Streaming AMF Connection). For testing, I
>increased the amount of data sent back to the Client when loading the
>list. Currently about 12000 Objects are returned using about 107MB
>(according to Adobe Scout).
>
>The returned list of objects is displayed in a Flexicious Ultimate
>DataGrid (The Performance of this is very sattisfying considering the
>amount of data it has to handle).
>
>There is a Feature to force a reload of the list. As removing all
>elements of an ArrayList and then adding 12000 Objects caused Major
>Performance problems, I changed the code to replace the source array of
>the ArrayCollection and then to make the list update itself after that.
>Each time the list is reloaded 107MB are used but I can't see any Memory
>being freed.
>
>Is there something wrong with my Approach? Do I have to Keep something
>else in mind? Are there some hidden internals keeping a reference to my
>objects so they aren't cleaned up?
>
>Help greatly appreciated.
>
>Chris