autosuggest component for lengthy xmls

[Nitin Gopi <ni...@gmail.com>]

I have a autosuggest library which I have been using in my project. I now
have a textinput where I have to use this library. But this time the input
has around 900,000 records. i.e, an xml having 900,000 nodes. When I used
it with the component, it works but it has become slow probably because of
filtering of the xmllistcollection on each key press. On each key press it
takes 2-3 sec for refreshing. Is there a way to speed up the process?
I tested this xml on the local machine.

-- 
You can only depend on yourself. The cavalry ain't coming.

Re: autosuggest component for lengthy xmls

[Scott Talsma <sc...@talsma.tv>]

I'll worked w/similar components.  One thing that helps is to have each
keypress schedule an autocomplete to take place in the future (e.g. in
100ms), and each keypress resets/restarts the timer.  That way, if the user
knows where they are going, you are not unnecessarily doing processing.

Also, does the Flex client really need to know about that much data?  I
would also consider reducing the the record size if you can.  An
autosuggest should is supposed to help the user, so returning hundreds of
rows that they have to scroll through defeats this.

Can you bust up the data in to a server-filtered dataset, which is further
refined by client-side logic?

For example, require a minimum number of characters to begin an
autocomplete search (e.g. 2+).  That lets your database reduce that 900k to
several thousand (or hundreds of rows?).  Maybe even better, have the db
routine first calculate the number of rows that your search string would
return, and if it exceeds a certain threshold  return 0 rows, w/an
information message: "60427 results pending").

Granted, this shifts a lot of load to the server/db.  I don't know if that
possibile in your scenario.  Also, if you can use AMF instead of XML, you
will cut down on deserialization time.

Once you have the results from an search that is initially filtered by the
server, you can use your existing client-side filterFunction() for further
refinement as additional characters are entered.


On Sun, Mar 31, 2013 at 9:40 AM, Nitin Gopi <ni...@gmail.com> wrote:

> I have a autosuggest library which I have been using in my project. I now
> have a textinput where I have to use this library. But this time the input
> has around 900,000 records. i.e, an xml having 900,000 nodes. When I used
> it with the component, it works but it has become slow probably because of
> filtering of the xmllistcollection on each key press. On each key press it
> takes 2-3 sec for refreshing. Is there a way to speed up the process?
> I tested this xml on the local machine.
>
> --
> You can only depend on yourself. The cavalry ain't coming.
>

Re: autosuggest component for lengthy xmls

[Mark Kessler <ke...@gmail.com>]

I believe the filterFunction goes over each item separately and treats each
item as an Object.  So if you had 900k records, it should be a consistent
(slow) speed for each key-press.  The more logic is in your filterFunction
the more compounded the effects you should see in the speed (slower with
more logic).

So if I can imagine sort of what you might be working with, I can see two
things that might help.

1.  The user should be able to keep typing without the list being
completely updated (filtering not completed).

2a.  A way to cancel the current filtering process when a user presses a
new key to start a new filter and not finish the old one.
or
2b.  The filtering shouldn't start until the user pauses their typing for a
set amount time before attempting to filter and produce a list.  This could
save on wasted processing until the user is looking for a suggestion.


Ultimately indexing would be more efficient or maybe breaking the giant
list down into multiple smaller (alphabet styled) lists to improve on
speed.  Either way maybe this email will get some other ideas going from
other people.


-Mark


On Sun, Mar 31, 2013 at 9:40 AM, Nitin Gopi <ni...@gmail.com> wrote:

> I have a autosuggest library which I have been using in my project. I now
> have a textinput where I have to use this library. But this time the input
> has around 900,000 records. i.e, an xml having 900,000 nodes. When I used
> it with the component, it works but it has become slow probably because of
> filtering of the xmllistcollection on each key press. On each key press it
> takes 2-3 sec for refreshing. Is there a way to speed up the process?
> I tested this xml on the local machine.
>
> --
> You can only depend on yourself. The cavalry ain't coming.
>

Re: autosuggest component for lengthy xmls

[Justin Mclean <ju...@classsoftware.com>]

Hi,

> When I used it with the component, it works but it has become slow probably because of
> filtering of the xmllistcollection on each key press.

It may depends on your filter function. XML operations are generally slower than operations on other types. You should see some speed improvement if you convert your XMLListCollection to an ArrayCollection of Strings.

As it runs the filter function on each item in your list every time and once filtered out an item will not show again in the list so adding a filter cache should improve speed as the user types. Reset the cache on focus in or backspace.

Something like this, untested and may not run as it,  but it should give you an idea of what to try.

private function filterLocation(item:Object):Boolean {
   value:String = item as String;

    if (filtered.hasOwnProperty(value)) {
	return false;    
     }

     |f (value.indexOf(userText.toLowerCase()) >= 0) {
         return true;
     }

     filtered[value] = true;

     return false;
}

Thanks,
Justin