You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by wactioner <jo...@webaction.com> on 2013/12/20 07:10:02 UTC
Lucene Search and Disk I/O
I had a similar question posted here
<http://lucene.472066.n3.nabble.com/Understanding-lucene-indexes-and-disk-I-O-td714698.html>
a few years earlier.
How does a lucene search handle disk I/O and reading from index files? I'm
not sure how much the previous answer stands true. I notice that some of the
index files referenced in the last answer (.tii files) don't exist anymore,
unless .tii files were renamed to .tip.
If anyone is familiar with the matter on this board, could you comment on
how ElasticSearch shards these files? Any thoughts would be appreciated.
Thank you!
--
View this message in context: http://lucene.472066.n3.nabble.com/Lucene-Search-and-Disk-I-O-tp4107540.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Search and Disk I/O
Posted by wactioner <jo...@webaction.com>.
Thanks for the response!
So it loads the terms index into memory, performs a binary search. Then
it'll use the index to read from the term dict like reading any other file,
relying on the OS's buffer cache. Makes sense.
--
View this message in context: http://lucene.472066.n3.nabble.com/Lucene-Search-and-Disk-I-O-tp4107540p4107650.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Search and Disk I/O
Posted by Michael McCandless <lu...@mikemccandless.com>.
The situation is pretty much the same as before, just different file names :)
Lucene pre-loads certain "hot" structures into RAM: the terms index
(.tip files), deleted documents (.del files), norms, etc.
For all other files, it relies on the OS to cache any hot pages in its
buffer/IO cache, and it typically reads sequentially so OS readahead
should work well.
Mike McCandless
http://blog.mikemccandless.com
On Fri, Dec 20, 2013 at 1:10 AM, wactioner <jo...@webaction.com> wrote:
> I had a similar question posted here
> <http://lucene.472066.n3.nabble.com/Understanding-lucene-indexes-and-disk-I-O-td714698.html>
> a few years earlier.
>
> How does a lucene search handle disk I/O and reading from index files? I'm
> not sure how much the previous answer stands true. I notice that some of the
> index files referenced in the last answer (.tii files) don't exist anymore,
> unless .tii files were renamed to .tip.
>
> If anyone is familiar with the matter on this board, could you comment on
> how ElasticSearch shards these files? Any thoughts would be appreciated.
> Thank you!
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Lucene-Search-and-Disk-I-O-tp4107540.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org