You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Bidisha Banerjee (Created) (JIRA)" <ji...@apache.org> on 2011/12/14 02:24:30 UTC
[jira] [Created] (SOLR-2967) StreamingUpdateSolrServer Error
StreamingUpdateSolrServer Error
-------------------------------
Key: SOLR-2967
URL: https://issues.apache.org/jira/browse/SOLR-2967
Project: Solr
Issue Type: Bug
Components: update
Affects Versions: 1.4
Environment: Unix
Reporter: Bidisha Banerjee
Priority: Minor
Hi,I am using Solr-1.4.0 version. Whenever I am trying you run a full Solr index, I get intermittent error as shown below and the numdocs in Statistics section does not reflect the exact count of index documents.(The amount of data is 1004384). Sometimes I get this error and sometimes not.
The queue size is 200 and thread is 20. I initialize StreamingUpdateSolr server as:
servers.add(new StreamingUpdateSolrServer(serverUrl,200,20));
I am using the following code for commit:
For StreamingUpdateSolrServer:
------------------------------
//UpdateRequest req;
//req.setAction(ACTION.COMMIT, true, false);
// StreamingUpdateSolrServer.request(req);
For SolrServer:
------------------
//UpdateResponse response = server.add(documents);
//UpdateResponse commitResp = server.commit();
The error:
---------------
ERROR StreamingUpdateSolrServer:259 - error
java.lang.Exception: Service Temporarily Unavailable
Service Temporarily Unavailable
request: http://<localhost>>/solr/testCOre/update
at org.apache.solr.client.solrj.impl.StreamingUpdateSolrServer$Runner.run(StreamingUpdateSolrServer.java:161)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
*************************************************************************************************
Also I get the following error in /update section of Solr admin page:
handlerStart : 1321903404556
requests : 68200
errors : 806
timeouts : 0
totalTime : 707909202
avgTimePerRequest : 10379.9
avgRequestsPerSecond : 0.0355129
************************************************************************************************
Sometimes I get this error:
12:24:13,066 INFO HttpMethodDirector:439 - I/O exception (java.net.SocketException) caught when processing request: Broken pipe
12:24:13,066 INFO HttpMethodDirector:445 - Retrying request
12:24:13,067 ERROR StreamingUpdateSolrServer:259 - error
org.apache.commons.httpclient.ProtocolException: Unbuffered entity enclosing request can not be repeated.
at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:487)
at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at org.apache.solr.client.solrj.impl.StreamingUpdateSolrServer$Runner.run(StreamingUpdateSolrServer.java:153)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
It would be great if you could let me know a probable fix for this. It is also difficult to debug as the error is intermittent.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2967) StreamingUpdateSolrServer Error
Posted by "Bidisha Banerjee (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169652#comment-13169652 ]
Bidisha Banerjee commented on SOLR-2967:
----------------------------------------
Hi Erick,
Thanks for the suggestion.I was not aware of adding the problem in Solr User List as I was unable to find my problem.
Actually my code is comitting at the end only. My only confusion regarding comitting in the code is using SolrConsumerAdapter whose consume() method is overriden and we do a commit in this section of code also by adding UpdateRequest.setAction(Action.commit,true,false) and then calling StreamingUpdateSolrServer.request(UpdateRequest).
Will removing the commit from this code help? Will using setCommitWithin() help us ?
> StreamingUpdateSolrServer Error
> -------------------------------
>
> Key: SOLR-2967
> URL: https://issues.apache.org/jira/browse/SOLR-2967
> Project: Solr
> Issue Type: Bug
> Components: update
> Affects Versions: 1.4
> Environment: Unix
> Reporter: Bidisha Banerjee
> Priority: Minor
>
> Hi,I am using Solr-1.4.0 version. Whenever I am trying you run a full Solr index, I get intermittent error as shown below and the numdocs in Statistics section does not reflect the exact count of index documents.(The amount of data is 1004384). Sometimes I get this error and sometimes not.
> The queue size is 200 and thread is 20. I initialize StreamingUpdateSolr server as:
> servers.add(new StreamingUpdateSolrServer(serverUrl,200,20));
> I am using the following code for commit:
> For StreamingUpdateSolrServer:
> ------------------------------
> //UpdateRequest req;
> //req.setAction(ACTION.COMMIT, true, false);
> // StreamingUpdateSolrServer.request(req);
> For SolrServer:
> ------------------
> //UpdateResponse response = server.add(documents);
> //UpdateResponse commitResp = server.commit();
> The error:
> ---------------
> ERROR StreamingUpdateSolrServer:259 - error
> java.lang.Exception: Service Temporarily Unavailable
> Service Temporarily Unavailable
> request: http://<localhost>>/solr/testCOre/update
> at org.apache.solr.client.solrj.impl.StreamingUpdateSolrServer$Runner.run(StreamingUpdateSolrServer.java:161)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> *************************************************************************************************
> Also I get the following error in /update section of Solr admin page:
> handlerStart : 1321903404556
> requests : 68200
> errors : 806
> timeouts : 0
> totalTime : 707909202
> avgTimePerRequest : 10379.9
> avgRequestsPerSecond : 0.0355129
> ************************************************************************************************
> Sometimes I get this error:
> 12:24:13,066 INFO HttpMethodDirector:439 - I/O exception (java.net.SocketException) caught when processing request: Broken pipe
> 12:24:13,066 INFO HttpMethodDirector:445 - Retrying request
> 12:24:13,067 ERROR StreamingUpdateSolrServer:259 - error
> org.apache.commons.httpclient.ProtocolException: Unbuffered entity enclosing request can not be repeated.
> at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:487)
> at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114)
> at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
> at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
> at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
> at org.apache.solr.client.solrj.impl.StreamingUpdateSolrServer$Runner.run(StreamingUpdateSolrServer.java:153)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> It would be great if you could let me know a probable fix for this. It is also difficult to debug as the error is intermittent.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2967) StreamingUpdateSolrServer Error
Posted by "Erick Erickson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169326#comment-13169326 ]
Erick Erickson commented on SOLR-2967:
--------------------------------------
This kind of thing is better raised on the Solr user's list before raising a Jira.
Offhand, I suspect you're shooting yourself in the foot. If you're committing after every add as your code seems to indicate, you could occasionally be triggering a very large merge that delays blocks other adds long enough to trigger a timeout on the client, resulting in the broken pipe and the assorted errors you're seeing.
Try removing the commit from your loop and setting your <autoCommit> values in your solrconfig.xml file to a reasonable value (say 300000 for 5 minutes). You can do a commit at the end of the entire run if you want.
> StreamingUpdateSolrServer Error
> -------------------------------
>
> Key: SOLR-2967
> URL: https://issues.apache.org/jira/browse/SOLR-2967
> Project: Solr
> Issue Type: Bug
> Components: update
> Affects Versions: 1.4
> Environment: Unix
> Reporter: Bidisha Banerjee
> Priority: Minor
>
> Hi,I am using Solr-1.4.0 version. Whenever I am trying you run a full Solr index, I get intermittent error as shown below and the numdocs in Statistics section does not reflect the exact count of index documents.(The amount of data is 1004384). Sometimes I get this error and sometimes not.
> The queue size is 200 and thread is 20. I initialize StreamingUpdateSolr server as:
> servers.add(new StreamingUpdateSolrServer(serverUrl,200,20));
> I am using the following code for commit:
> For StreamingUpdateSolrServer:
> ------------------------------
> //UpdateRequest req;
> //req.setAction(ACTION.COMMIT, true, false);
> // StreamingUpdateSolrServer.request(req);
> For SolrServer:
> ------------------
> //UpdateResponse response = server.add(documents);
> //UpdateResponse commitResp = server.commit();
> The error:
> ---------------
> ERROR StreamingUpdateSolrServer:259 - error
> java.lang.Exception: Service Temporarily Unavailable
> Service Temporarily Unavailable
> request: http://<localhost>>/solr/testCOre/update
> at org.apache.solr.client.solrj.impl.StreamingUpdateSolrServer$Runner.run(StreamingUpdateSolrServer.java:161)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> *************************************************************************************************
> Also I get the following error in /update section of Solr admin page:
> handlerStart : 1321903404556
> requests : 68200
> errors : 806
> timeouts : 0
> totalTime : 707909202
> avgTimePerRequest : 10379.9
> avgRequestsPerSecond : 0.0355129
> ************************************************************************************************
> Sometimes I get this error:
> 12:24:13,066 INFO HttpMethodDirector:439 - I/O exception (java.net.SocketException) caught when processing request: Broken pipe
> 12:24:13,066 INFO HttpMethodDirector:445 - Retrying request
> 12:24:13,067 ERROR StreamingUpdateSolrServer:259 - error
> org.apache.commons.httpclient.ProtocolException: Unbuffered entity enclosing request can not be repeated.
> at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:487)
> at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114)
> at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
> at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
> at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
> at org.apache.solr.client.solrj.impl.StreamingUpdateSolrServer$Runner.run(StreamingUpdateSolrServer.java:153)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> It would be great if you could let me know a probable fix for this. It is also difficult to debug as the error is intermittent.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (SOLR-2967) StreamingUpdateSolrServer Error
Posted by "Erick Erickson (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erick Erickson resolved SOLR-2967.
----------------------------------
Resolution: Not A Problem
Since this is against 1.4 there's no chance it'll be fixed in that version, plus I think the user can avoid the problem by not committing as often.
Bidisha:
If my suggestion doesn't work, please raise the issue on the user's list.
> StreamingUpdateSolrServer Error
> -------------------------------
>
> Key: SOLR-2967
> URL: https://issues.apache.org/jira/browse/SOLR-2967
> Project: Solr
> Issue Type: Bug
> Components: update
> Affects Versions: 1.4
> Environment: Unix
> Reporter: Bidisha Banerjee
> Priority: Minor
>
> Hi,I am using Solr-1.4.0 version. Whenever I am trying you run a full Solr index, I get intermittent error as shown below and the numdocs in Statistics section does not reflect the exact count of index documents.(The amount of data is 1004384). Sometimes I get this error and sometimes not.
> The queue size is 200 and thread is 20. I initialize StreamingUpdateSolr server as:
> servers.add(new StreamingUpdateSolrServer(serverUrl,200,20));
> I am using the following code for commit:
> For StreamingUpdateSolrServer:
> ------------------------------
> //UpdateRequest req;
> //req.setAction(ACTION.COMMIT, true, false);
> // StreamingUpdateSolrServer.request(req);
> For SolrServer:
> ------------------
> //UpdateResponse response = server.add(documents);
> //UpdateResponse commitResp = server.commit();
> The error:
> ---------------
> ERROR StreamingUpdateSolrServer:259 - error
> java.lang.Exception: Service Temporarily Unavailable
> Service Temporarily Unavailable
> request: http://<localhost>>/solr/testCOre/update
> at org.apache.solr.client.solrj.impl.StreamingUpdateSolrServer$Runner.run(StreamingUpdateSolrServer.java:161)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> *************************************************************************************************
> Also I get the following error in /update section of Solr admin page:
> handlerStart : 1321903404556
> requests : 68200
> errors : 806
> timeouts : 0
> totalTime : 707909202
> avgTimePerRequest : 10379.9
> avgRequestsPerSecond : 0.0355129
> ************************************************************************************************
> Sometimes I get this error:
> 12:24:13,066 INFO HttpMethodDirector:439 - I/O exception (java.net.SocketException) caught when processing request: Broken pipe
> 12:24:13,066 INFO HttpMethodDirector:445 - Retrying request
> 12:24:13,067 ERROR StreamingUpdateSolrServer:259 - error
> org.apache.commons.httpclient.ProtocolException: Unbuffered entity enclosing request can not be repeated.
> at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:487)
> at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114)
> at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
> at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
> at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
> at org.apache.solr.client.solrj.impl.StreamingUpdateSolrServer$Runner.run(StreamingUpdateSolrServer.java:153)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:595)
> It would be great if you could let me know a probable fix for this. It is also difficult to debug as the error is intermittent.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org