You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Sai Gadde (JIRA)" <ji...@apache.org> on 2013/10/30 02:48:25 UTC
[jira] [Updated] (SOLR-5402) CLONE - SolrCloud 4.5 bulk add errors
[ https://issues.apache.org/jira/browse/SOLR-5402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sai Gadde updated SOLR-5402:
----------------------------
Description:
We use out of the box Solr 4.5.1 no customization done. If we merge documents via SolrJ to a single server it is perfectly working fine.
But as soon as we add another node to the cloud we are getting following while merging documents. We merge about 500 at a time using SolrJ. These 500 documents in total are about few MB (1-3) in size.
This is the error we are getting on the server (10.10.10.116 - IP is irrelavent just for clarity)where merging is happening. 10.10.10.119 is the new node here. This server gets RemoteSolrException
shard update error StdNode: http://10.10.10.119:8980/solr/mycore/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [1,12468]
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:425)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
at org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
On the other server 10.10.10.119 we get following error
org.apache.solr.common.SolrException: Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [1,12468]
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.ctc.wstx.exc.WstxParsingException: Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [1,12369]
at com.ctc.wstx.sr.StreamScanner.constructWfcException(StreamScanner.java:630)
at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:461)
at com.ctc.wstx.sr.BasicStreamReader.handleExtraRoot(BasicStreamReader.java:2155)
at com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2070)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2647)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1019)
at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:213)
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
... 22 more
We tried with 4.5.0 first and then with 4.5.1 later. Both produce exact same error.
Note: This error is not present when we switched exact setup to 4.4.0
was:
Since Solr 4.5 bulk adding documents via SolrJ (at least) is causing errors.
// build array list of SolrInputDocuments
server.add(docs);
I've tried with CUSS (which swallows exceptions as expected) however they are shown in the logs on server, and with CloudSolrServer which is returning the errors as well as seeing them in the server logs.
I've tried downgrading my SolrJ to 4.4, still errors so looks like a regression in the server code. Reverting to Solr 4.4 on server and I don't get errors (however run into deadlock issues).
I raised this issue in IRC - NOT the mailing list, and elyorag suggested opening a ticket, and to mention this has now been discussed in IRC.
The exceptions would indicate I'm attempting to do multiple operations in a single request which is malformed. I am not, I am only attempting to add documents.
Stack traces seen here:
14:57:13 ERROR SolrCmdDistributor shard update error RetryNode: http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Illegal to have multiple roots (start tag in epilog?).
shard update error RetryNode: http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [18,327]
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
at org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
----
org.apache.solr.common.SolrException: Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [7,6314]
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: com.ctc.wstx.exc.WstxParsingException: Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [7,6314]
at com.ctc.wstx.sr.StreamScanner.constructWfcException(StreamScanner.java:630)
at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:461)
at com.ctc.wstx.sr.BasicStreamReader.handleExtraRoot(BasicStreamReader.java:2155)
at com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2070)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2647)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1019)
at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:213)
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
... 22 more
> CLONE - SolrCloud 4.5 bulk add errors
> -------------------------------------
>
> Key: SOLR-5402
> URL: https://issues.apache.org/jira/browse/SOLR-5402
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
> Affects Versions: 4.5
> Reporter: Sai Gadde
> Fix For: 4.5.1
>
>
> We use out of the box Solr 4.5.1 no customization done. If we merge documents via SolrJ to a single server it is perfectly working fine.
> But as soon as we add another node to the cloud we are getting following while merging documents. We merge about 500 at a time using SolrJ. These 500 documents in total are about few MB (1-3) in size.
> This is the error we are getting on the server (10.10.10.116 - IP is irrelavent just for clarity)where merging is happening. 10.10.10.119 is the new node here. This server gets RemoteSolrException
> shard update error StdNode: http://10.10.10.119:8980/solr/mycore/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Illegal to have multiple roots (start tag in epilog?).
> at [row,col {unknown-source}]: [1,12468]
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:425)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> at org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
> at org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:1)
> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> On the other server 10.10.10.119 we get following error
> org.apache.solr.common.SolrException: Illegal to have multiple roots (start tag in epilog?).
> at [row,col {unknown-source}]: [1,12468]
> at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176)
> at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
> at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
> at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
> at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
> at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: com.ctc.wstx.exc.WstxParsingException: Illegal to have multiple roots (start tag in epilog?).
> at [row,col {unknown-source}]: [1,12369]
> at com.ctc.wstx.sr.StreamScanner.constructWfcException(StreamScanner.java:630)
> at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:461)
> at com.ctc.wstx.sr.BasicStreamReader.handleExtraRoot(BasicStreamReader.java:2155)
> at com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2070)
> at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2647)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1019)
> at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:213)
> at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
> ... 22 more
> We tried with 4.5.0 first and then with 4.5.1 later. Both produce exact same error.
> Note: This error is not present when we switched exact setup to 4.4.0
--
This message was sent by Atlassian JIRA
(v6.1#6144)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org