Improving disk efficiency for autocompleter / spellchecker

[Elmer <ev...@gmail.com>]

Hello again :)

I've created a autocompleter class, based on the contrib SpellChecker
source code (it creates start1-20 grams, no middle/end grams). The use
case for this class, as with most autocompleters, is that it is called
on each keypress from the user.
What happens is, that on each keypress, a static method
'autocompleteOnField' is called, which creates a new instance of
AutoCompleter:
autoCompleter = new AutoCompleter(FSDirectory.open(new
File(pathToIndex)));

It has a finalize block where it calls the close method of the
AutoCompleter (works the same as SpellChecker). In practice, this means
that every time a key is hit in the search field, my HDD led blinks.
Since I'd like to have an efficient program and reduce stress on the
hard drives as much as possible, I'd like to know if there are some
tricks to reduce this usage.
I already tried using a RAMDirectory instead of FSDirectory, but this
eats up too much memory (out-of-mem exception). 

I'm currently changing the code to have a single instance of the
AutoCompleter that is reused every time 'autocompleteOnField' is called
-> FSDirectory.open(new File(pathToIndex)) and autocompleter.close()
will only be invoked once (until the index is rebuilt/updated), which
will probably reduce the IO reads/writes.

Are there more ways to improve disk efficiency?

BR,
Elmer


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Improving disk efficiency for autocompleter / spellchecker

[Ian Lea <ia...@gmail.com>]

Not opening the index for every keypress should make a big difference!

Most of  the tips in
http://wiki.apache.org/lucene-java/ImproveSearchingSpeed will apply.


--
Ian.


On Fri, Jun 10, 2011 at 1:52 PM, Elmer <ev...@gmail.com> wrote:
> Hello again :)
>
> I've created a autocompleter class, based on the contrib SpellChecker
> source code (it creates start1-20 grams, no middle/end grams). The use
> case for this class, as with most autocompleters, is that it is called
> on each keypress from the user.
> What happens is, that on each keypress, a static method
> 'autocompleteOnField' is called, which creates a new instance of
> AutoCompleter:
> autoCompleter = new AutoCompleter(FSDirectory.open(new
> File(pathToIndex)));
>
> It has a finalize block where it calls the close method of the
> AutoCompleter (works the same as SpellChecker). In practice, this means
> that every time a key is hit in the search field, my HDD led blinks.
> Since I'd like to have an efficient program and reduce stress on the
> hard drives as much as possible, I'd like to know if there are some
> tricks to reduce this usage.
> I already tried using a RAMDirectory instead of FSDirectory, but this
> eats up too much memory (out-of-mem exception).
>
> I'm currently changing the code to have a single instance of the
> AutoCompleter that is reused every time 'autocompleteOnField' is called
> -> FSDirectory.open(new File(pathToIndex)) and autocompleter.close()
> will only be invoked once (until the index is rebuilt/updated), which
> will probably reduce the IO reads/writes.
>
> Are there more ways to improve disk efficiency?
>
> BR,
> Elmer
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org