IndexReade​rWarmer and IndexWrite​r.GetReade​r()

[Shubhanshu Pathak <sh...@gmail.com>]

Hello Group Members,

I am a new user of Lucene. Currently I am using lucene.net 3.0.3.

 In my application I am trying to get IndexReader from the existing
instance of IndexWriter.

Like this...
IndexReader indexReader = m_IndexWriter.GetReader();

However when I read the documentation of the GetReader() API it states that
it consumes extra resources like RAM, CPU time etc. The same I experienced
as well. There is a solution for this in terms of "pre-warming". However I
didnt get the concept of it fully. Since you have used IndexReaderWarmer
abstract class and used it for the 'MergedSegmentWarmer' so if you can
throw some light on what the Warm() method of this concrete class should
implement then it will be great help for me.

internal class CustomIndexReaderWarmer : IndexWriter.IndexReaderWarmer
    {
        public override void Warm(IndexReader reader)
        {
           // Here what???
        }
    }

Also when the 'MergedSegmentWarmer' property needs to be set i.e.
after the call of the GetReader() or before that?

IndexReader indexReader = m_IndexWriter.GetReader();

m_IndexWriter.MergedSegmentWarmer = new CustomIndexReaderWarmer();



OR



m_IndexWriter.MergedSegmentWarmer = new CustomIndexReaderWarmer();

IndexReader indexReader = m_IndexWriter.GetReader();



It will also be great to know

1. how 'IndexReaderWarmer' helps in improving the performance?

2. As I saw in the call stack Call of IW.GetReader() is actually
flushing the segements and making a big segment. Then where the warmer
comes in the picture?

3. Is there any other way apart from the IW.GetReader() to get the
latest reader which contains both committed and un-committed data?

4. Is there any other way apart from 'warming' the index to achieve a
better performance while calling IW.GetReader()

Thanks & Regards,

Shubhanshu

Re: IndexReade​rWarmer and IndexWrite​r.GetReade​r()

[Michael McCandless <lu...@mikemccandless.com>]

The warmer is just used to warm merged segments before IW.getReader "sees" them.

The idea is the a merge could produce a very large segment, and it
could be you want to warm up the OS's IO cache on that segment before
it becomes visible to a newly opened IW.getReader.

It may improve the performance of IW.getReader in that the OS will
have things like live docs and terms index "hot" if you've warmed them
... but the impact likely won't be as large as you are expecting ...

Mike McCandless

http://blog.mikemccandless.com


On Wed, Aug 20, 2014 at 4:59 AM, Shubhanshu Pathak
<sh...@gmail.com> wrote:
> Hello Group Members,
>
> I am a new user of Lucene. Currently I am using lucene.net 3.0.3.
>
>  In my application I am trying to get IndexReader from the existing
> instance of IndexWriter.
>
> Like this...
> IndexReader indexReader = m_IndexWriter.GetReader();
>
> However when I read the documentation of the GetReader() API it states that
> it consumes extra resources like RAM, CPU time etc. The same I experienced
> as well. There is a solution for this in terms of "pre-warming". However I
> didnt get the concept of it fully. Since you have used IndexReaderWarmer
> abstract class and used it for the 'MergedSegmentWarmer' so if you can
> throw some light on what the Warm() method of this concrete class should
> implement then it will be great help for me.
>
> internal class CustomIndexReaderWarmer : IndexWriter.IndexReaderWarmer
>     {
>         public override void Warm(IndexReader reader)
>         {
>            // Here what???
>         }
>     }
>
> Also when the 'MergedSegmentWarmer' property needs to be set i.e.
> after the call of the GetReader() or before that?
>
> IndexReader indexReader = m_IndexWriter.GetReader();
>
> m_IndexWriter.MergedSegmentWarmer = new CustomIndexReaderWarmer();
>
>
>
> OR
>
>
>
> m_IndexWriter.MergedSegmentWarmer = new CustomIndexReaderWarmer();
>
> IndexReader indexReader = m_IndexWriter.GetReader();
>
>
>
> It will also be great to know
>
> 1. how 'IndexReaderWarmer' helps in improving the performance?
>
> 2. As I saw in the call stack Call of IW.GetReader() is actually
> flushing the segements and making a big segment. Then where the warmer
> comes in the picture?
>
> 3. Is there any other way apart from the IW.GetReader() to get the
> latest reader which contains both committed and un-committed data?
>
> 4. Is there any other way apart from 'warming' the index to achieve a
> better performance while calling IW.GetReader()
>
> Thanks & Regards,
>
> Shubhanshu

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org