You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by navdeep agarwal <na...@gmail.com> on 2014/07/05 17:40:47 UTC

error during heavy indexing

i am getting following error on heavy indexing .i am using Solr 4.7
.creating index in hdfs through map reduce .sending docs in batch of 500000
.

ERROR org.apache.solr.core.SolrCore  – java.lang.RuntimeException: [was
class org.eclipse.jetty.io.EofException] early EOF
    at
com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
    at com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
    at
com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657)
    at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
    at org.apache.solr.handler.loader.XMLLoader.readDoc(XMLLoader.java:397)
    at
org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:246)
    at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
    at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
    at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
    at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1916)
    at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:768)
    at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415)
    at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:205)
    at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
    at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
    at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
    at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
    at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
    at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
    at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
    at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
    at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:368)
    at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
    at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
    at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
    at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
    at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
    at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
    at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:744)
Caused by: org.eclipse.jetty.io.EofException: early EOF
    at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:65)
    at java.io.InputStream.read(InputStream.java:101)
    at com.ctc.wstx.io.UTF8Reader.loadMore(UTF8Reader.java:365)
    at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:110)
    at com.ctc.wstx.io.MergedReader.read(MergedReader.java:101)
    at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84)
    at
com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57)
    at com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:992)
    at
com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReader.java:4628)
    at
com.ctc.wstx.sr.BasicStreamReader.readCoalescedText(BasicStreamReader.java:4126)
    at
com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:3701)
    at
com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3649)
    ... 36 more

5409264 [qtp667019593-1270] ERROR
org.apache.solr.servlet.SolrDispatchFilter  –
null:java.lang.RuntimeException: [was class
org.eclipse.jetty.io.EofException] early EOF
    at
com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
    at com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
    at
com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657)
    at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
    at org.apache.solr.handler.loader.XMLLoader.readDoc(XMLLoader.java:397)
    at
org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:246)
    at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
    at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
    at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
    at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1916)
    at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:768)
    at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415)
    at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:205)
    at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
    at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
    at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
    at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
    at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
    at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
    at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
    at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
    at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:368)
    at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
    at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
    at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
    at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
    at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
    at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
    at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:744)
Caused by: org.eclipse.jetty.io.EofException: early EOF
    at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:65)
    at java.io.InputStream.read(InputStream.java:101)
    at com.ctc.wstx.io.UTF8Reader.loadMore(UTF8Reader.java:365)
    at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:110)
    at com.ctc.wstx.io.MergedReader.read(MergedReader.java:101)
    at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84)
    at
com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57)
    at com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:992)
    at
com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReader.java:4628)
    at
com.ctc.wstx.sr.BasicStreamReader.readCoalescedText(BasicStreamReader.java:4126)
    at
com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:3701)
    at
com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3649)
    ... 36 more

Re: error during heavy indexing

Posted by navdeep710 <na...@gmail.com>.

thank you Shawn for replying.
its the client that is doing the indexing(queries).i have set connection
timeout and connection socket timeout as 0 explicitly .i am doing indexing
through MapReduce jobs . so indexing is happening in multi thread
environment .






--
View this message in context: http://lucene.472066.n3.nabble.com/error-during-heavy-indexing-tp4145725p4145779.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: error during heavy indexing

Posted by Shawn Heisey <so...@elyograg.org>.

On 7/5/2014 9:40 AM, navdeep agarwal wrote:
> i am getting following error on heavy indexing .i am using Solr 4.7
> .creating index in hdfs through map reduce .sending docs in batch of 500000
> .
> 
> ERROR org.apache.solr.core.SolrCore  – java.lang.RuntimeException: [was
> class org.eclipse.jetty.io.EofException] early EOF

This means that your client (the software making the HTTP connections)
disconnected before Solr was finished with the request, most likely due
to a configured connection inactivity timeout.  It's actually Jetty that
reports the situation, since it is handling the low-level TCP layers.

I don't know enough about the stacktraces to know whether this was the
client doing the indexing, or whether it was a client doing queries.  If
I had to guess, I would say it was likely a client doing queries.  Your
queries are likely taking longer during heavy indexing, long enough to
exceed the client timeout.

Thanks,
Shawn