You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Peter Matthew Eichman <pe...@umd.edu> on 2017/02/14 16:57:33 UTC
Getting "Error getting file length for [segments_5]" warnings in Solr 6.4.0
Hello all,
I am running Solr 6.4.0, and while I am attempting to index my Fedora 4
data, I keep getting warning messages in my solr.log: "WARN
(qtp401424608-18) [ x:fedora4] o.a.s.h.a.LukeRequestHandler Error
getting file length for [segments_5]". And after that, the indexing stops,
and the core is left in a non-current state until I issue a manual commit
request to it.
What might be causing this behavior? We did migrate this core from Solr 4
to Solr 6, so I was thinking it might be related to some config setting
that works in 4 but messes up in 6?
Here is the full stack trace from the log:
2017-02-14 16:51:11.429 WARN (qtp401424608-18) [ x:fedora4]
o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_5]
java.nio.file.NoSuchFileException:
/apps/solr/solr64/cores/fedora4/data/index/segments_5
at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at
sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at
sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at
sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.Files.size(Files.java:2332)
at
org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243)
at
org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:128)
at
org.apache.lucene.store.FilterDirectory.fileLength(FilterDirectory.java:67)
at
org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(LukeRequestHandler.java:614)
at
org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:587)
at
org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(LukeRequestHandler.java:138)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:166)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2306)
at
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:296)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
Thanks,
-Peter
--
Peter Eichman
Senior Software Developer
University of Maryland Libraries
peichman@umd.edu
Re: Getting "Error getting file length for [segments_5]" warnings in
Solr 6.4.0
Posted by Peter Matthew Eichman <pe...@umd.edu>.
Shawn,
We have confirmed that yes, this is just log noise, and possibly related to
the admin interface and not the actual indexing process. As for indexing
stopping, that was not actually the case. I got confused about how many
documents were in the collection I was indexing, and thought there should
have been more.
Thanks,
-Peter
On Tue, Feb 14, 2017 at 6:10 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 2/14/2017 9:57 AM, Peter Matthew Eichman wrote:
> > I am running Solr 6.4.0, and while I am attempting to index my Fedora
> > 4 data, I keep getting warning messages in my solr.log: "WARN
> > (qtp401424608-18) [ x:fedora4] o.a.s.h.a.LukeRequestHandler Error
> > getting file length for [segments_5]". And after that, the indexing
> > stops, and the core is left in a non-current state until I issue a
> > manual commit request to it.
>
> I have seen this warning frequently on newer Solr versions. It doesn't
> seem to actually affect Solr's operation, it just results in a lot of
> excess logging. It is unlikely to cause issues with indexing.
>
> In your statement, what does "the indexing stops" mean? This is quite
> vague about what's actually happening. Error messages related to the
> *indexing* have not been provided. Are your indexing clients receiving
> any error messages?
>
> Thanks,
> Shawn
>
>
--
Peter Eichman
Senior Software Developer
University of Maryland Libraries
peichman@umd.edu
Re: Getting "Error getting file length for [segments_5]" warnings in
Solr 6.4.0
Posted by Shawn Heisey <ap...@elyograg.org>.
On 2/14/2017 9:57 AM, Peter Matthew Eichman wrote:
> I am running Solr 6.4.0, and while I am attempting to index my Fedora
> 4 data, I keep getting warning messages in my solr.log: "WARN
> (qtp401424608-18) [ x:fedora4] o.a.s.h.a.LukeRequestHandler Error
> getting file length for [segments_5]". And after that, the indexing
> stops, and the core is left in a non-current state until I issue a
> manual commit request to it.
I have seen this warning frequently on newer Solr versions. It doesn't
seem to actually affect Solr's operation, it just results in a lot of
excess logging. It is unlikely to cause issues with indexing.
In your statement, what does "the indexing stops" mean? This is quite
vague about what's actually happening. Error messages related to the
*indexing* have not been provided. Are your indexing clients receiving
any error messages?
Thanks,
Shawn
Re: Getting "Error getting file length for [segments_5]" warnings in
Solr 6.4.0
Posted by Peter Matthew Eichman <pe...@umd.edu>.
It is not. This is happening for other segment files, not just segment_5,
too.
On Tue, Feb 14, 2017 at 12:04 PM, David Hastings <
hastings.recursive@gmail.com> wrote:
> well, if you look at the error:
> NoSuchFileException
>
> its looking for the segment file for generation 5, are you sure its
> present?
>
> On Tue, Feb 14, 2017 at 11:57 AM, Peter Matthew Eichman <pe...@umd.edu>
> wrote:
>
> > Hello all,
> >
> > I am running Solr 6.4.0, and while I am attempting to index my Fedora 4
> > data, I keep getting warning messages in my solr.log: "WARN
> > (qtp401424608-18) [ x:fedora4] o.a.s.h.a.LukeRequestHandler Error
> > getting file length for [segments_5]". And after that, the indexing
> stops,
> > and the core is left in a non-current state until I issue a manual commit
> > request to it.
> >
> > What might be causing this behavior? We did migrate this core from Solr 4
> > to Solr 6, so I was thinking it might be related to some config setting
> > that works in 4 but messes up in 6?
> >
> > Here is the full stack trace from the log:
> >
> > 2017-02-14 16:51:11.429 WARN (qtp401424608-18) [ x:fedora4]
> > o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_5]
> > java.nio.file.NoSuchFileException:
> > /apps/solr/solr64/cores/fedora4/data/index/segments_5
> > at
> > sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> > at
> > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> > at
> > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> > at
> > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(
> > UnixFileAttributeViews.java:55)
> > at
> > sun.nio.fs.UnixFileSystemProvider.readAttributes(
> > UnixFileSystemProvider.java:144)
> > at
> > sun.nio.fs.LinuxFileSystemProvider.readAttributes(
> > LinuxFileSystemProvider.java:99)
> > at java.nio.file.Files.readAttributes(Files.java:1737)
> > at java.nio.file.Files.size(Files.java:2332)
> > at
> > org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243)
> > at
> > org.apache.lucene.store.NRTCachingDirectory.fileLength(
> > NRTCachingDirectory.java:128)
> > at
> > org.apache.lucene.store.FilterDirectory.fileLength(
> > FilterDirectory.java:67)
> > at
> > org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(
> > LukeRequestHandler.java:614)
> > at
> > org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(
> > LukeRequestHandler.java:587)
> > at
> > org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(
> > LukeRequestHandler.java:138)
> > at
> > org.apache.solr.handler.RequestHandlerBase.handleRequest(
> > RequestHandlerBase.java:166)
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2306)
> > at
> > org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658)
> > at org.apache.solr.servlet.HttpSolrCall.call(
> > HttpSolrCall.java:464)
> > at
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> > SolrDispatchFilter.java:345)
> > at
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> > SolrDispatchFilter.java:296)
> > at
> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> > doFilter(ServletHandler.java:1691)
> > at
> > org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:582)
> > at
> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
> > ScopedHandler.java:143)
> > at
> > org.eclipse.jetty.security.SecurityHandler.handle(
> > SecurityHandler.java:548)
> > at
> > org.eclipse.jetty.server.session.SessionHandler.
> > doHandle(SessionHandler.java:226)
> > at
> > org.eclipse.jetty.server.handler.ContextHandler.
> > doHandle(ContextHandler.java:1180)
> > at
> > org.eclipse.jetty.servlet.ServletHandler.doScope(
> ServletHandler.java:512)
> > at
> > org.eclipse.jetty.server.session.SessionHandler.
> > doScope(SessionHandler.java:185)
> > at
> > org.eclipse.jetty.server.handler.ContextHandler.
> > doScope(ContextHandler.java:1112)
> > at
> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
> > ScopedHandler.java:141)
> > at
> > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
> > ContextHandlerCollection.java:213)
> > at
> > org.eclipse.jetty.server.handler.HandlerCollection.
> > handle(HandlerCollection.java:119)
> > at
> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> > HandlerWrapper.java:134)
> > at org.eclipse.jetty.server.Server.handle(Server.java:534)
> > at org.eclipse.jetty.server.HttpChannel.handle(
> > HttpChannel.java:320)
> > at
> > org.eclipse.jetty.server.HttpConnection.onFillable(
> > HttpConnection.java:251)
> > at
> > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(
> > AbstractConnection.java:273)
> > at org.eclipse.jetty.io.FillInterest.fillable(
> > FillInterest.java:95)
> > at
> > org.eclipse.jetty.io.SelectChannelEndPoint$2.run(
> > SelectChannelEndPoint.java:93)
> > at
> > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
> > executeProduceConsume(ExecuteProduceConsume.java:303)
> > at
> > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
> > produceConsume(ExecuteProduceConsume.java:148)
> > at
> > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(
> > ExecuteProduceConsume.java:136)
> > at
> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> > QueuedThreadPool.java:671)
> > at
> > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
> > QueuedThreadPool.java:589)
> > at java.lang.Thread.run(Thread.java:745)
> >
> > Thanks,
> > -Peter
> >
> > --
> > Peter Eichman
> > Senior Software Developer
> > University of Maryland Libraries
> > peichman@umd.edu
> >
>
--
Peter Eichman
Senior Software Developer
University of Maryland Libraries
peichman@umd.edu
Re: Getting "Error getting file length for [segments_5]" warnings in
Solr 6.4.0
Posted by David Hastings <ha...@gmail.com>.
well, if you look at the error:
NoSuchFileException
its looking for the segment file for generation 5, are you sure its present?
On Tue, Feb 14, 2017 at 11:57 AM, Peter Matthew Eichman <pe...@umd.edu>
wrote:
> Hello all,
>
> I am running Solr 6.4.0, and while I am attempting to index my Fedora 4
> data, I keep getting warning messages in my solr.log: "WARN
> (qtp401424608-18) [ x:fedora4] o.a.s.h.a.LukeRequestHandler Error
> getting file length for [segments_5]". And after that, the indexing stops,
> and the core is left in a non-current state until I issue a manual commit
> request to it.
>
> What might be causing this behavior? We did migrate this core from Solr 4
> to Solr 6, so I was thinking it might be related to some config setting
> that works in 4 but messes up in 6?
>
> Here is the full stack trace from the log:
>
> 2017-02-14 16:51:11.429 WARN (qtp401424608-18) [ x:fedora4]
> o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_5]
> java.nio.file.NoSuchFileException:
> /apps/solr/solr64/cores/fedora4/data/index/segments_5
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(
> UnixFileAttributeViews.java:55)
> at
> sun.nio.fs.UnixFileSystemProvider.readAttributes(
> UnixFileSystemProvider.java:144)
> at
> sun.nio.fs.LinuxFileSystemProvider.readAttributes(
> LinuxFileSystemProvider.java:99)
> at java.nio.file.Files.readAttributes(Files.java:1737)
> at java.nio.file.Files.size(Files.java:2332)
> at
> org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243)
> at
> org.apache.lucene.store.NRTCachingDirectory.fileLength(
> NRTCachingDirectory.java:128)
> at
> org.apache.lucene.store.FilterDirectory.fileLength(
> FilterDirectory.java:67)
> at
> org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(
> LukeRequestHandler.java:614)
> at
> org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(
> LukeRequestHandler.java:587)
> at
> org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(
> LukeRequestHandler.java:138)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(
> RequestHandlerBase.java:166)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2306)
> at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658)
> at org.apache.solr.servlet.HttpSolrCall.call(
> HttpSolrCall.java:464)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:345)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:296)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> doFilter(ServletHandler.java:1691)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(
> SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1180)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
> at
> org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:1112)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
> ContextHandlerCollection.java:213)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.
> handle(HandlerCollection.java:119)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:534)
> at org.eclipse.jetty.server.HttpChannel.handle(
> HttpChannel.java:320)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(
> HttpConnection.java:251)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(
> AbstractConnection.java:273)
> at org.eclipse.jetty.io.FillInterest.fillable(
> FillInterest.java:95)
> at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(
> SelectChannelEndPoint.java:93)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
> executeProduceConsume(ExecuteProduceConsume.java:303)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
> produceConsume(ExecuteProduceConsume.java:148)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(
> ExecuteProduceConsume.java:136)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> QueuedThreadPool.java:671)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
> QueuedThreadPool.java:589)
> at java.lang.Thread.run(Thread.java:745)
>
> Thanks,
> -Peter
>
> --
> Peter Eichman
> Senior Software Developer
> University of Maryland Libraries
> peichman@umd.edu
>