Re: Best way to returning hits after search?

[Mohammad Norouzi <mn...@gmail.com>]

Hello
I am implemented an IndexResultSet just like java.sql.ResultSet with all its
methods. when I call searcher.search(...) I pass a the returned Hits to my
IndexResultSet.
in the IndexResultSet I have getString(String) getString(int) getInt()
next() previous() absolute() and all methods of the java.sql.ResultSet
besides, because I am using MyFaces in my application, I customized
DataModel in order to support pagination and I keep open my reader so,
pagination works fine.
in addition, I provided a SercherPool to keep readers open and close when
the user ends his searching or an idle time occured.


On 3/1/07, Doron Cohen <DO...@il.ibm.com> wrote:
>
> Antony Bowesman <ad...@teamware.com> wrote on 27/02/2007 17:37:41:
>
> > Doron Cohen wrote:
> > > The collect() method is going to be invoked once for each document
> that
> > > matches the query (having nonzero score). If the index is very large,
> that
> > > may turn to be a very large number of calls. Often, search
> applications
> > > only fetch additional data (doc fields) for only a small subset of the
> > > entire set of documents matching a query - e.g. first page (0-9),
> second
> > > page (10-19), etc.  But if your application is going to fetch in an
> > > exhaustive manner, and especially for a short field like DB_ID, it
> > > sometimes makes sense to cache in memory the entire field (its values
> for
> > > all the docs), for the entire life of the index reader/searcher, and
> use
> > > that cached data. The collect method can then use that cached data.
> >
> > That's an excellent idea!  We cannot easily change our client
> > implementation, so
> > have to support the exhaustive retrieval for now, although I do limit
> the
>
> > absolute max hits that will be returned.  We are hoping to implement
> > paging in a
> > later client version.
> >
> > I'm not sure I can cache all the GUIDs though.  A GUID is 20 bytes
> > and there are
> > two that need to be cached.  The document count could be up to
> 100M,though in
> > most cases <20M.  I am keeping a BitSet filter cache for a searcher for
> each
> > user's mail, so I could extend that to cache all the IDs for that
> > user and give
> > that cache a shortish life and/or limit the total cache available.
> > That would
> > really help.
> >
> > I'll have a play - thanks for the input.
> > Antony
>
> If you decide to cache stored field value in memory, FieldCache may be
> useful for this - so you don't have to implement your own cache - you can
> access the field values with something like:
>    FieldCache fieldCache = FieldCache.DEFAULT;
>    String db_id_field[] =
> fieldCache.getStrings(indexReader,"DB_ID_FIELD_NAME");
> Those values are valid for the lifetime of the index-reader. Once a new
> index reader is opened, when GC collects the unused old index reader
> object, it would also be able to collect (from the cache) unused field
> values.
>
> See also http://www.gossamer-threads.com/lists/lucene/java-user/39352
>
> Doron
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


-- 
Regards,
Mohammad

RE: Performance in having Multiple Index files

["Mordo, Aviran (EXP N-NANNATEK)" <av...@lmco.com>]

Yes, it will affect the search performance because you need to merge the
results from the different indexes. The best performance is from a
single index. The more indexes you have the more time it takes to
search.

Aviran
http://www.aviransplace.com 

-----Original Message-----
From: Raaj [mailto:raajwizz@yahoo.com] 
Sent: Thursday, March 01, 2007 2:50 AM
To: java-user@lucene.apache.org
Subject: Performance in having Multiple Index files

hi all,

i have requirement where in i create an index file for each xml file . i
have over 100/150 xml files which are all related . 

if create 100/150 index files and query using these indices , will this
affect the performance of the search operation . 

bye
raaj



 
---------------------------------
Need a quick answer? Get one in minutes from people who know. Ask your
question on Yahoo! Answers.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Performance in having Multiple Index files

[Raaj <ra...@yahoo.com>]

hi all,

i have requirement where in i create an index file for each xml file . i have over 100/150 xml files which are all related . 

if create 100/150 index files and query using these indices , will this affect the performance of the search operation . 

bye
raaj



 
---------------------------------
Need a quick answer? Get one in minutes from people who know. Ask your question on Yahoo! Answers.