You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Mossaab Bagdouri <ba...@yahoo.fr> on 2012/08/26 20:40:09 UTC
Lucene 4.0-BETA : MultiReader isCurrent openIfChanged
Hi,
I've just migrated my webapp from Lucene 3.6 to 4.0-BETA. My 2 indexes are
updated every couple of minutes by a batch. The webapp searcher needs to
get refreshed whenever this happens. In 3.6, I was doing it this way:
private IndexSearcher getIndexSearcher() {
try {
if (is == null || !is.getIndexReader().isCurrent()) {
if (is != null) {
is.close();
}
IndexReader readers[] = new IndexReader[2];
for (int i = 0; i < 2; i++) {
readers[i] = IndexReader.open(MyFSDirectories.get(i));
}
is = new IndexSearcher(new MultiReader(readers));
}
} catch (Exception e) {
e.printStackTrace();
}
return is;
}
Now that the API changed in 4.0-BETA. I'm doing it this odd way:
private IndexSearcher getIndexSearcher() {
try {
boolean refresh = false;
if (is == null) {
refresh = true;
} else {
MultiReader ir = ((MultiReader) is.getIndexReader());
List<DirectoryReader> list = (List<DirectoryReader>)
ir.getSequentialSubReaders();
try {
if (!list.get(0).isCurrent() ||
!list.get(1).isCurrent()) {
refresh = true;
}
} catch (Exception e) {
e.printStackTrace();
refresh = true;
}
}
if (refresh) {
if (is != null) {
((MultiReader) is.getIndexReader()).close();
}
DirectoryReader readers[] = new DirectoryReader[2];
for (int i = 0; i < 2; i++) {
readers[i] =
DirectoryReader.open(MyFSDirectories.get(i));
}
is = new IndexSearcher(new MultiReader(readers));
}
} catch (Exception e) {
e.printStackTrace();
}
return is;
}
The problem is that sometimes isCurrent() returns an exception indicating
that the underlying IndexReader is closed.
My questions are:
1. How can I verify of the IndexReader is open before calling isCurrent();
2. Is there a better way to deal with MultiReaders?
Regards,
Mossaab
Re: Lucene 4.0-BETA : MultiReader isCurrent openIfChanged
Posted by Vitaly Funstein <vf...@gmail.com>.
You have probably figured it out by now, but my suggestion would be to use
SearcherManager the way it is documented for maintaining a searcher backed
by an NRT reader.
On Sun, Aug 26, 2012 at 2:03 PM, Mossaab Bagdouri <bagdouri_mossaab@yahoo.fr
> wrote:
> Thanks for the quick reply.
>
> I've changed my code to the following. The problem now is that the
> MultiReader doesn't seem to get closed. In fact, the number of open files
> (returned by "lsof | grep index/ | wc -l") keeps increasing whenever the
> IndexWriter adds new documents until the webapp crashes. Am I missing
> something?
>
> private IndexSearcher getIndexSearcher() {
> try {
> boolean refresh = false;
> if (is == null) {
> refresh = true;
> } else {
> MultiReader ir = ((MultiReader) is.getIndexReader());
> if (ir.getRefCount() == 0) {
> refresh = true;
> } else {
> for (DirectoryReader dr : (List<DirectoryReader>)
> ir.getSequentialSubReaders()) {
> if (!dr.isCurrent()) {
> refresh = true;
> ir.close();
> break;
> }
> }
> }
> }
>
> if (refresh) {
> DirectoryReader readers[] = new DirectoryReader[2];
> for (int i = 0; i < 2; i++) {
> readers[i] =
> DirectoryReader.open(MyFSDirectories.get(i));
> }
> is = new IndexSearcher(new MultiReader(readers, true));
> }
> } catch (Exception e) {
> e.printStackTrace();
> }
> return is;
> }
>
> Regards,
> Mossaab
>
> 2012/8/26 Uwe Schindler <uw...@thetaphi.de>
>
> > 1. getRefCount()
> > 2. No
> >
> >
> >
> > Mossaab Bagdouri <ba...@yahoo.fr> schrieb:
> >
> > >Hi,
> > >
> > >I've just migrated my webapp from Lucene 3.6 to 4.0-BETA. My 2 indexes
> > >are
> > >updated every couple of minutes by a batch. The webapp searcher needs
> > >to
> > >get refreshed whenever this happens. In 3.6, I was doing it this way:
> > >
> > >private IndexSearcher getIndexSearcher() {
> > > try {
> > > if (is == null || !is.getIndexReader().isCurrent()) {
> > > if (is != null) {
> > > is.close();
> > > }
> > > IndexReader readers[] = new IndexReader[2];
> > > for (int i = 0; i < 2; i++) {
> > > readers[i] = IndexReader.open(MyFSDirectories.get(i));
> > > }
> > > is = new IndexSearcher(new MultiReader(readers));
> > > }
> > > } catch (Exception e) {
> > > e.printStackTrace();
> > > }
> > > return is;
> > > }
> > >
> > >Now that the API changed in 4.0-BETA. I'm doing it this odd way:
> > >
> > >private IndexSearcher getIndexSearcher() {
> > > try {
> > > boolean refresh = false;
> > > if (is == null) {
> > > refresh = true;
> > > } else {
> > > MultiReader ir = ((MultiReader) is.getIndexReader());
> > > List<DirectoryReader> list = (List<DirectoryReader>)
> > >ir.getSequentialSubReaders();
> > > try {
> > > if (!list.get(0).isCurrent() ||
> > >!list.get(1).isCurrent()) {
> > > refresh = true;
> > > }
> > > } catch (Exception e) {
> > > e.printStackTrace();
> > > refresh = true;
> > > }
> > >
> > > }
> > > if (refresh) {
> > > if (is != null) {
> > > ((MultiReader) is.getIndexReader()).close();
> > > }
> > > DirectoryReader readers[] = new DirectoryReader[2];
> > > for (int i = 0; i < 2; i++) {
> > > readers[i] =
> > >DirectoryReader.open(MyFSDirectories.get(i));
> > > }
> > > is = new IndexSearcher(new MultiReader(readers));
> > > }
> > > } catch (Exception e) {
> > > e.printStackTrace();
> > > }
> > > return is;
> > > }
> > >
> > >The problem is that sometimes isCurrent() returns an exception
> > >indicating
> > >that the underlying IndexReader is closed.
> > >
> > >My questions are:
> > >1. How can I verify of the IndexReader is open before calling
> > >isCurrent();
> > >2. Is there a better way to deal with MultiReaders?
> > >
> > >Regards,
> > >Mossaab
> >
> > --
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, 28213 Bremen
> > http://www.thetaphi.de
>
Re: Lucene 4.0-BETA : MultiReader isCurrent openIfChanged
Posted by Mossaab Bagdouri <ba...@yahoo.fr>.
Thanks for the quick reply.
I've changed my code to the following. The problem now is that the
MultiReader doesn't seem to get closed. In fact, the number of open files
(returned by "lsof | grep index/ | wc -l") keeps increasing whenever the
IndexWriter adds new documents until the webapp crashes. Am I missing
something?
private IndexSearcher getIndexSearcher() {
try {
boolean refresh = false;
if (is == null) {
refresh = true;
} else {
MultiReader ir = ((MultiReader) is.getIndexReader());
if (ir.getRefCount() == 0) {
refresh = true;
} else {
for (DirectoryReader dr : (List<DirectoryReader>)
ir.getSequentialSubReaders()) {
if (!dr.isCurrent()) {
refresh = true;
ir.close();
break;
}
}
}
}
if (refresh) {
DirectoryReader readers[] = new DirectoryReader[2];
for (int i = 0; i < 2; i++) {
readers[i] =
DirectoryReader.open(MyFSDirectories.get(i));
}
is = new IndexSearcher(new MultiReader(readers, true));
}
} catch (Exception e) {
e.printStackTrace();
}
return is;
}
Regards,
Mossaab
2012/8/26 Uwe Schindler <uw...@thetaphi.de>
> 1. getRefCount()
> 2. No
>
>
>
> Mossaab Bagdouri <ba...@yahoo.fr> schrieb:
>
> >Hi,
> >
> >I've just migrated my webapp from Lucene 3.6 to 4.0-BETA. My 2 indexes
> >are
> >updated every couple of minutes by a batch. The webapp searcher needs
> >to
> >get refreshed whenever this happens. In 3.6, I was doing it this way:
> >
> >private IndexSearcher getIndexSearcher() {
> > try {
> > if (is == null || !is.getIndexReader().isCurrent()) {
> > if (is != null) {
> > is.close();
> > }
> > IndexReader readers[] = new IndexReader[2];
> > for (int i = 0; i < 2; i++) {
> > readers[i] = IndexReader.open(MyFSDirectories.get(i));
> > }
> > is = new IndexSearcher(new MultiReader(readers));
> > }
> > } catch (Exception e) {
> > e.printStackTrace();
> > }
> > return is;
> > }
> >
> >Now that the API changed in 4.0-BETA. I'm doing it this odd way:
> >
> >private IndexSearcher getIndexSearcher() {
> > try {
> > boolean refresh = false;
> > if (is == null) {
> > refresh = true;
> > } else {
> > MultiReader ir = ((MultiReader) is.getIndexReader());
> > List<DirectoryReader> list = (List<DirectoryReader>)
> >ir.getSequentialSubReaders();
> > try {
> > if (!list.get(0).isCurrent() ||
> >!list.get(1).isCurrent()) {
> > refresh = true;
> > }
> > } catch (Exception e) {
> > e.printStackTrace();
> > refresh = true;
> > }
> >
> > }
> > if (refresh) {
> > if (is != null) {
> > ((MultiReader) is.getIndexReader()).close();
> > }
> > DirectoryReader readers[] = new DirectoryReader[2];
> > for (int i = 0; i < 2; i++) {
> > readers[i] =
> >DirectoryReader.open(MyFSDirectories.get(i));
> > }
> > is = new IndexSearcher(new MultiReader(readers));
> > }
> > } catch (Exception e) {
> > e.printStackTrace();
> > }
> > return is;
> > }
> >
> >The problem is that sometimes isCurrent() returns an exception
> >indicating
> >that the underlying IndexReader is closed.
> >
> >My questions are:
> >1. How can I verify of the IndexReader is open before calling
> >isCurrent();
> >2. Is there a better way to deal with MultiReaders?
> >
> >Regards,
> >Mossaab
>
> --
> Uwe Schindler
> H.-H.-Meier-Allee 63, 28213 Bremen
> http://www.thetaphi.de
Re: Lucene 4.0-BETA : MultiReader isCurrent openIfChanged
Posted by Uwe Schindler <uw...@thetaphi.de>.
1. getRefCount()
2. No
Mossaab Bagdouri <ba...@yahoo.fr> schrieb:
>Hi,
>
>I've just migrated my webapp from Lucene 3.6 to 4.0-BETA. My 2 indexes
>are
>updated every couple of minutes by a batch. The webapp searcher needs
>to
>get refreshed whenever this happens. In 3.6, I was doing it this way:
>
>private IndexSearcher getIndexSearcher() {
> try {
> if (is == null || !is.getIndexReader().isCurrent()) {
> if (is != null) {
> is.close();
> }
> IndexReader readers[] = new IndexReader[2];
> for (int i = 0; i < 2; i++) {
> readers[i] = IndexReader.open(MyFSDirectories.get(i));
> }
> is = new IndexSearcher(new MultiReader(readers));
> }
> } catch (Exception e) {
> e.printStackTrace();
> }
> return is;
> }
>
>Now that the API changed in 4.0-BETA. I'm doing it this odd way:
>
>private IndexSearcher getIndexSearcher() {
> try {
> boolean refresh = false;
> if (is == null) {
> refresh = true;
> } else {
> MultiReader ir = ((MultiReader) is.getIndexReader());
> List<DirectoryReader> list = (List<DirectoryReader>)
>ir.getSequentialSubReaders();
> try {
> if (!list.get(0).isCurrent() ||
>!list.get(1).isCurrent()) {
> refresh = true;
> }
> } catch (Exception e) {
> e.printStackTrace();
> refresh = true;
> }
>
> }
> if (refresh) {
> if (is != null) {
> ((MultiReader) is.getIndexReader()).close();
> }
> DirectoryReader readers[] = new DirectoryReader[2];
> for (int i = 0; i < 2; i++) {
> readers[i] =
>DirectoryReader.open(MyFSDirectories.get(i));
> }
> is = new IndexSearcher(new MultiReader(readers));
> }
> } catch (Exception e) {
> e.printStackTrace();
> }
> return is;
> }
>
>The problem is that sometimes isCurrent() returns an exception
>indicating
>that the underlying IndexReader is closed.
>
>My questions are:
>1. How can I verify of the IndexReader is open before calling
>isCurrent();
>2. Is there a better way to deal with MultiReaders?
>
>Regards,
>Mossaab
--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de