You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by re...@nlplabs.co.in on 2012/01/27 04:40:11 UTC
Issue on reading Lucene Index stored on NFS mounted disk on Ubuntu
Hi,
Needed help on an issue that is faced when accessing the same Lucene index
on a NFS mounted Unbutu server through different machines configured to
access the same NFS mounted disk.
S/W used:
Lucene: lucene-core-3.0.3
Tomcat : Tomcat 6.0
OS: Ubuntu
1) The same NFS mounted fiel was able to read/write into the same file
through a java program from different m/c in the cluster pointing to the
same NFS mounted disk.
2) But when creating and accessing Lucene Index files on the same NFS
mounted file from two different machines in the cluster caused error
provided below:
Scenario:
* The application reading/writing a Lucene Index on the NFS mounted disk
is hosted on two Tomcat Application server.
* Through one of the App server the Lucene Index is written.
* The same data is able to be retrieved from both the app server. No issues
* Now through one of the App Server updated the existing Lucene Index with
newer data.
* the updated Lucene data is able to be retieved from the same app server
through which update was made.
* But retrieving the same information through the other App Server gives
the error mentioned below. Sometime Read EOF error thrown for the same
scenario.
Could somebody throw some light on what could be the issue? Thanks in
advance.
Error Snippet:
org.apache.lucene.index.CorruptIndexException: did not read all bytes from
file "_1k.fnm": read 1 vs size 185
at org.apache.lucene.index.FieldInfos.read(FieldInfos.java:367)
at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:71)
at
org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:120)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:605)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:583)
at
org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:109)
at
org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27)
at
org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:79)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:677)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:73)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:316)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:202)
at org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:63)
at
com.nlplabs.index.IndexCreatorFactory.searchWithinIndex(IndexCreatorFactory.java:902)
at com.nlplabs.index.IndexCreatorFactory$searchWithinIndex.call(Unknown
Source)
at SearchController.renderHistoryPage(SearchController.groovy:892)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1058)
at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1070)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1003)
at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1070)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886)
at
org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
at SearchController$_closure7.doCall(SearchController.groovy:981)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
Regards,
Reno Rayan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Issue on reading Lucene Index stored on NFS mounted disk on Ubuntu
Posted by Michael McCandless <lu...@mikemccandless.com>.
Are you using a custom IndexDeletionPolicy? (Generally this is
necessary w/ NFS, since it does not protect still-open files from
being deleted, unlike most local filesystems).
If not... to do a quick test, pass in NoDeletionPolicy, and see if it
fixes the issues you're seeing? (You obviously can't go into
production like that -- it deletes nothing!).
If that fixes it then you should make a more sane deletion policy (eg,
delete a commit once it's been obsoleted more than N minutes/hours
ago, as long as you can ensure all readers will have reopened by that
time).
Mike McCandless
http://blog.mikemccandless.com
On Thu, Jan 26, 2012 at 10:40 PM, <re...@nlplabs.co.in> wrote:
> Hi,
>
> Needed help on an issue that is faced when accessing the same Lucene index
> on a NFS mounted Unbutu server through different machines configured to
> access the same NFS mounted disk.
>
> S/W used:
> Lucene: lucene-core-3.0.3
> Tomcat : Tomcat 6.0
> OS: Ubuntu
>
> 1) The same NFS mounted fiel was able to read/write into the same file
> through a java program from different m/c in the cluster pointing to the
> same NFS mounted disk.
>
> 2) But when creating and accessing Lucene Index files on the same NFS
> mounted file from two different machines in the cluster caused error
> provided below:
>
> Scenario:
> * The application reading/writing a Lucene Index on the NFS mounted disk
> is hosted on two Tomcat Application server.
> * Through one of the App server the Lucene Index is written.
> * The same data is able to be retrieved from both the app server. No issues
>
> * Now through one of the App Server updated the existing Lucene Index with
> newer data.
> * the updated Lucene data is able to be retieved from the same app server
> through which update was made.
> * But retrieving the same information through the other App Server gives
> the error mentioned below. Sometime Read EOF error thrown for the same
> scenario.
>
> Could somebody throw some light on what could be the issue? Thanks in
> advance.
>
> Error Snippet:
> org.apache.lucene.index.CorruptIndexException: did not read all bytes from
> file "_1k.fnm": read 1 vs size 185
> at org.apache.lucene.index.FieldInfos.read(FieldInfos.java:367)
> at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:71)
> at
> org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:120)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:605)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:583)
> at
> org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:109)
> at
> org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27)
> at
> org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:79)
> at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:677)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:73)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:316)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:202)
> at org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:63)
> at
> com.nlplabs.index.IndexCreatorFactory.searchWithinIndex(IndexCreatorFactory.java:902)
> at com.nlplabs.index.IndexCreatorFactory$searchWithinIndex.call(Unknown
> Source)
> at SearchController.renderHistoryPage(SearchController.groovy:892)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
> at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
> at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1058)
> at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1070)
> at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886)
> at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1003)
> at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1070)
> at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886)
> at
> org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
> at SearchController$_closure7.doCall(SearchController.groovy:981)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
>
> Regards,
> Reno Rayan
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org