You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by sp...@gmx.eu on 2012/06/05 12:51:12 UTC
ReadTimeout on commit
Hi,
I'm indexing documents in batches of 100 docs. Then commit.
Sometimes I get this exception:
org.apache.solr.client.solrj.SolrServerException:
java.net.SocketTimeoutException: Read timed out
at
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpS
olrServer.java:475)
at
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpS
olrServer.java:249)
at
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractU
pdateRequest.java:105)
at
org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:178)
I found some similar postings in the web, all recommending autocommit. This
is unfortunately not an option for me, because I have to know whether solr
committed or not.
What is causing this timeout?
I'm using these settings in solrj:
server.setSoTimeout(1000);
server.setConnectionTimeout(100);
server.setDefaultMaxConnectionsPerHost(100);
server.setMaxTotalConnections(100);
server.setFollowRedirects(false);
server.setAllowCompression(true);
server.setMaxRetries(1);
Thank you
RE: ReadTimeout on commit
Posted by sp...@gmx.eu.
Hi Jack, hi Erik,
thanks for the tips! It's solr 3.6
I increased the batch to 1000 docs and the timeout to 10 s. Now it works.
And I will implement the retry around the commit-call.
Thx!
> -----Original Message-----
> From: Jack Krupansky [mailto:jack@basetechnology.com]
> Sent: Mittwoch, 6. Juni 2012 13:52
> To: solr-user@lucene.apache.org
> Subject: Re: ReadTimeout on commit
>
> As Erick says, you are probably hitting an occasional
> automatic background
> merge which takes a bit longer. That is not an indication of
> a problem.
> Increase your connection timeout. Check the log to see how
> long the merge or
> "slow commit" takes. You have a timeout of 1000 which is 1
> second. Make it
> longer, and possibly put the commit or other indexing
> operations in a loop
> with a few retries before considering connection timeout a
> fatal error.
> Occasional delays are a fact or life in a multi-process, networked
> environment.
>
> -- Jack Krupansky
>
> -----Original Message-----
> From: Erick Erickson
> Sent: Wednesday, June 06, 2012 7:02 AM
> To: solr-user@lucene.apache.org
> Subject: Re: ReadTimeout on commit
>
> You're probably hitting a background merge and the request is timing
> out even though the commit succeeds. Try querying for the data in
> the last packet to test this.
>
> And you don't say what version of Solr you're using.
>
> One test you can do is increase the number of documents before
> a commit. If merging is the problem I'd expect you to _still_
> encounter
> this problem, just much less often. That would at least tell
> you if this
> is the right path to investigate.
>
> Best
> Erick
>
> On Tue, Jun 5, 2012 at 6:51 AM, <sp...@gmx.eu> wrote:
> > Hi,
> >
> > I'm indexing documents in batches of 100 docs. Then commit.
> >
> > Sometimes I get this exception:
> >
> > org.apache.solr.client.solrj.SolrServerException:
> > java.net.SocketTimeoutException: Read timed out
> > at
> >
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.reques
> t(CommonsHttpS
> > olrServer.java:475)
> > at
> >
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.reques
> t(CommonsHttpS
> > olrServer.java:249)
> > at
> >
> org.apache.solr.client.solrj.request.AbstractUpdateRequest.pro
> cess(AbstractU
> > pdateRequest.java:105)
> > at
> > org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:178)
> >
> >
> > I found some similar postings in the web, all recommending
> autocommit.
> > This
> > is unfortunately not an option for me, because I have to
> know whether solr
> > committed or not.
> >
> > What is causing this timeout?
> >
> > I'm using these settings in solrj:
> >
> > server.setSoTimeout(1000);
> > server.setConnectionTimeout(100);
> > server.setDefaultMaxConnectionsPerHost(100);
> > server.setMaxTotalConnections(100);
> > server.setFollowRedirects(false);
> > server.setAllowCompression(true);
> > server.setMaxRetries(1);
> >
> > Thank you
> >
>
Re: ReadTimeout on commit
Posted by Jack Krupansky <ja...@basetechnology.com>.
As Erick says, you are probably hitting an occasional automatic background
merge which takes a bit longer. That is not an indication of a problem.
Increase your connection timeout. Check the log to see how long the merge or
"slow commit" takes. You have a timeout of 1000 which is 1 second. Make it
longer, and possibly put the commit or other indexing operations in a loop
with a few retries before considering connection timeout a fatal error.
Occasional delays are a fact or life in a multi-process, networked
environment.
-- Jack Krupansky
-----Original Message-----
From: Erick Erickson
Sent: Wednesday, June 06, 2012 7:02 AM
To: solr-user@lucene.apache.org
Subject: Re: ReadTimeout on commit
You're probably hitting a background merge and the request is timing
out even though the commit succeeds. Try querying for the data in
the last packet to test this.
And you don't say what version of Solr you're using.
One test you can do is increase the number of documents before
a commit. If merging is the problem I'd expect you to _still_ encounter
this problem, just much less often. That would at least tell you if this
is the right path to investigate.
Best
Erick
On Tue, Jun 5, 2012 at 6:51 AM, <sp...@gmx.eu> wrote:
> Hi,
>
> I'm indexing documents in batches of 100 docs. Then commit.
>
> Sometimes I get this exception:
>
> org.apache.solr.client.solrj.SolrServerException:
> java.net.SocketTimeoutException: Read timed out
> at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpS
> olrServer.java:475)
> at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpS
> olrServer.java:249)
> at
> org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractU
> pdateRequest.java:105)
> at
> org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:178)
>
>
> I found some similar postings in the web, all recommending autocommit.
> This
> is unfortunately not an option for me, because I have to know whether solr
> committed or not.
>
> What is causing this timeout?
>
> I'm using these settings in solrj:
>
> server.setSoTimeout(1000);
> server.setConnectionTimeout(100);
> server.setDefaultMaxConnectionsPerHost(100);
> server.setMaxTotalConnections(100);
> server.setFollowRedirects(false);
> server.setAllowCompression(true);
> server.setMaxRetries(1);
>
> Thank you
>
Re: ReadTimeout on commit
Posted by Erick Erickson <er...@gmail.com>.
You're probably hitting a background merge and the request is timing
out even though the commit succeeds. Try querying for the data in
the last packet to test this.
And you don't say what version of Solr you're using.
One test you can do is increase the number of documents before
a commit. If merging is the problem I'd expect you to _still_ encounter
this problem, just much less often. That would at least tell you if this
is the right path to investigate.
Best
Erick
On Tue, Jun 5, 2012 at 6:51 AM, <sp...@gmx.eu> wrote:
> Hi,
>
> I'm indexing documents in batches of 100 docs. Then commit.
>
> Sometimes I get this exception:
>
> org.apache.solr.client.solrj.SolrServerException:
> java.net.SocketTimeoutException: Read timed out
> at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpS
> olrServer.java:475)
> at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpS
> olrServer.java:249)
> at
> org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractU
> pdateRequest.java:105)
> at
> org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:178)
>
>
> I found some similar postings in the web, all recommending autocommit. This
> is unfortunately not an option for me, because I have to know whether solr
> committed or not.
>
> What is causing this timeout?
>
> I'm using these settings in solrj:
>
> server.setSoTimeout(1000);
> server.setConnectionTimeout(100);
> server.setDefaultMaxConnectionsPerHost(100);
> server.setMaxTotalConnections(100);
> server.setFollowRedirects(false);
> server.setAllowCompression(true);
> server.setMaxRetries(1);
>
> Thank you
>
Re: ReadTimeout on commit
Posted by Mark Miller <ma...@gmail.com>.
Looks like the commit is taking longer than your set timeout.
On Jun 5, 2012, at 6:51 AM, <sp...@gmx.eu> <sp...@gmx.eu> wrote:
> Hi,
>
> I'm indexing documents in batches of 100 docs. Then commit.
>
> Sometimes I get this exception:
>
> org.apache.solr.client.solrj.SolrServerException:
> java.net.SocketTimeoutException: Read timed out
> at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpS
> olrServer.java:475)
> at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpS
> olrServer.java:249)
> at
> org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractU
> pdateRequest.java:105)
> at
> org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:178)
>
>
> I found some similar postings in the web, all recommending autocommit. This
> is unfortunately not an option for me, because I have to know whether solr
> committed or not.
>
> What is causing this timeout?
>
> I'm using these settings in solrj:
>
> server.setSoTimeout(1000);
> server.setConnectionTimeout(100);
> server.setDefaultMaxConnectionsPerHost(100);
> server.setMaxTotalConnections(100);
> server.setFollowRedirects(false);
> server.setAllowCompression(true);
> server.setMaxRetries(1);
>
> Thank you
>
- Mark Miller
lucidimagination.com