highlight optimization

[Stelios Eliakis <el...@gmail.com>]

Hi,
I have a collection of 500 txt documents and I implement a web
application(JSP) for searching these documents.
In addition, the application shows the BestFragment of each result and
highlights the query terms.
My application is slow enough (about 2,5-3 seconds for each query) even if I
run it from my computer (It's not publiched yet).
Do you suggest me something in order to improve speed?
I have read that you have to keep the indexsearcher open. Is it right? and
how could I do that(when I must close it)?

In Highlighting I use the following
                     String result =
highlighter.getBestFragment(tokenStream,text)


text parameter must be String so I open the document and convert it to
String. Of course it is time consuming. Is there a different way?

Thanks in advance,

-- 
Stelios Eliakis

Re: highlight optimization

[Erick Erickson <er...@gmail.com>]

The fastest way to see if opening/closing your searcher is a problem would
be to write a tiny little program that opened the index, fired off a few
queries and timed each one. The queries can be canned, of course. I'm
thinking this is, say, less that 20 lines (including imports). If you're
familiar with junit, think of it in those terms.

Once you've proved that is a bottleneck you want to work on, you probably
need a search server. We've used XmlRpc, which has server code built-in, and
has worked like a charm for us. It'll add a bit of complexity, but it'll
keep your searchers open.

That said, I suspect that someone will chime in with another solution, since
this is already implemented <G>....

Erick

On 10/9/06, Stelios Eliakis <el...@gmail.com> wrote:
>
> Hi,
> I have a collection of 500 txt documents and I implement a web
> application(JSP) for searching these documents.
> In addition, the application shows the BestFragment of each result and
> highlights the query terms.
> My application is slow enough (about 2,5-3 seconds for each query) even if
> I
> run it from my computer (It's not publiched yet).
> Do you suggest me something in order to improve speed?
> I have read that you have to keep the indexsearcher open. Is it right? and
> how could I do that(when I must close it)?
>
> In Highlighting I use the following
>                      String result =
> highlighter.getBestFragment(tokenStream,text)
>
>
> text parameter must be String so I open the document and convert it to
> String. Of course it is time consuming. Is there a different way?
>
> Thanks in advance,
>
> --
> Stelios Eliakis
>
>