You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Max Lynch <ih...@gmail.com> on 2010/07/12 18:59:27 UTC
CommonsHttpSolrServer add document hangs
Hey guys,
I'm using Solr 1.4.1 and I've been having some problems lately with code
that adds documents through a CommonsHttpSolrServer. It seems that randomly
the call to theserver.add() will hang. I am currently running my code in a
single thread, but I noticed this would happen in multi threaded code as
well. The jar version of commons-httpclient is 3.1.
I got a thread dump of the process, and one thread seems to be waiting on
the org.apache.commons.httpclient.MultiThreadedHttpConnectionManager as
shown below. All other threads are in a RUNNABLE state (besides the
Finalizer daemon).
[java] Full thread dump Java HotSpot(TM) 64-Bit Server VM (16.3-b01
mixed mode):
[java]
[java] "MultiThreadedHttpConnectionManager cleanup" daemon prio=10
tid=0x00007f441051c800 nid=0x527c in Object.wait() [0x00007f4417e2f000]
[java] java.lang.Thread.State: WAITING (on object monitor)
[java] at java.lang.Object.wait(Native Method)
[java] - waiting on <0x00007f443ae5b290> (a
java.lang.ref.ReferenceQueue$Lock)
[java] at
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
[java] - locked <0x00007f443ae5b290> (a
java.lang.ref.ReferenceQueue$Lock)
[java] at
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
[java] at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122)
Any ideas?
Thanks.
RE: CommonsHttpSolrServer add document hangs
Posted by Robert Petersen <ro...@buy.com>.
You could try a master slave setup using replication perhaps, then the
slave serves searches and indexing commits on the master won't hang up
searches at least...
Here is the description: http://wiki.apache.org/solr/SolrReplication
-----Original Message-----
From: Max Lynch [mailto:ihasmax@gmail.com]
Sent: Monday, July 12, 2010 11:57 AM
To: solr-user@lucene.apache.org
Subject: Re: CommonsHttpSolrServer add document hangs
Thanks Robert,
My script did start going again, but it was waiting for about half an
hour
which seems a bit excessive to me. Is there some tuning I can do on the
solr end to optimize for my use case, which is very heavy on commits and
very light on searches (I do most of my searches on the raw Lucene index
in
the background)?
Thanks.
On Mon, Jul 12, 2010 at 12:06 PM, Robert Petersen <ro...@buy.com>
wrote:
> Maybe solr is busy doing a commit or optimize?
>
> -----Original Message-----
> From: Max Lynch [mailto:ihasmax@gmail.com]
> Sent: Monday, July 12, 2010 9:59 AM
> To: solr-user@lucene.apache.org
> Subject: CommonsHttpSolrServer add document hangs
>
> Hey guys,
> I'm using Solr 1.4.1 and I've been having some problems lately with
code
> that adds documents through a CommonsHttpSolrServer. It seems that
> randomly
> the call to theserver.add() will hang. I am currently running my code
> in a
> single thread, but I noticed this would happen in multi threaded code
as
> well. The jar version of commons-httpclient is 3.1.
>
> I got a thread dump of the process, and one thread seems to be waiting
> on
> the org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
as
> shown below. All other threads are in a RUNNABLE state (besides the
> Finalizer daemon).
>
> [java] Full thread dump Java HotSpot(TM) 64-Bit Server VM
(16.3-b01
> mixed mode):
> [java]
> [java] "MultiThreadedHttpConnectionManager cleanup" daemon prio=10
> tid=0x00007f441051c800 nid=0x527c in Object.wait()
[0x00007f4417e2f000]
> [java] java.lang.Thread.State: WAITING (on object monitor)
> [java] at java.lang.Object.wait(Native Method)
> [java] - waiting on <0x00007f443ae5b290> (a
> java.lang.ref.ReferenceQueue$Lock)
> [java] at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
> [java] - locked <0x00007f443ae5b290> (a
> java.lang.ref.ReferenceQueue$Lock)
> [java] at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
> [java] at
>
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$Referen
> ceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122)
>
> Any ideas?
>
> Thanks.
>
Re: CommonsHttpSolrServer add document hangs
Posted by Max Lynch <ih...@gmail.com>.
Thanks Robert,
My script did start going again, but it was waiting for about half an hour
which seems a bit excessive to me. Is there some tuning I can do on the
solr end to optimize for my use case, which is very heavy on commits and
very light on searches (I do most of my searches on the raw Lucene index in
the background)?
Thanks.
On Mon, Jul 12, 2010 at 12:06 PM, Robert Petersen <ro...@buy.com> wrote:
> Maybe solr is busy doing a commit or optimize?
>
> -----Original Message-----
> From: Max Lynch [mailto:ihasmax@gmail.com]
> Sent: Monday, July 12, 2010 9:59 AM
> To: solr-user@lucene.apache.org
> Subject: CommonsHttpSolrServer add document hangs
>
> Hey guys,
> I'm using Solr 1.4.1 and I've been having some problems lately with code
> that adds documents through a CommonsHttpSolrServer. It seems that
> randomly
> the call to theserver.add() will hang. I am currently running my code
> in a
> single thread, but I noticed this would happen in multi threaded code as
> well. The jar version of commons-httpclient is 3.1.
>
> I got a thread dump of the process, and one thread seems to be waiting
> on
> the org.apache.commons.httpclient.MultiThreadedHttpConnectionManager as
> shown below. All other threads are in a RUNNABLE state (besides the
> Finalizer daemon).
>
> [java] Full thread dump Java HotSpot(TM) 64-Bit Server VM (16.3-b01
> mixed mode):
> [java]
> [java] "MultiThreadedHttpConnectionManager cleanup" daemon prio=10
> tid=0x00007f441051c800 nid=0x527c in Object.wait() [0x00007f4417e2f000]
> [java] java.lang.Thread.State: WAITING (on object monitor)
> [java] at java.lang.Object.wait(Native Method)
> [java] - waiting on <0x00007f443ae5b290> (a
> java.lang.ref.ReferenceQueue$Lock)
> [java] at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
> [java] - locked <0x00007f443ae5b290> (a
> java.lang.ref.ReferenceQueue$Lock)
> [java] at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
> [java] at
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$Referen
> ceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122)
>
> Any ideas?
>
> Thanks.
>
RE: CommonsHttpSolrServer add document hangs
Posted by Robert Petersen <ro...@buy.com>.
Maybe solr is busy doing a commit or optimize?
-----Original Message-----
From: Max Lynch [mailto:ihasmax@gmail.com]
Sent: Monday, July 12, 2010 9:59 AM
To: solr-user@lucene.apache.org
Subject: CommonsHttpSolrServer add document hangs
Hey guys,
I'm using Solr 1.4.1 and I've been having some problems lately with code
that adds documents through a CommonsHttpSolrServer. It seems that
randomly
the call to theserver.add() will hang. I am currently running my code
in a
single thread, but I noticed this would happen in multi threaded code as
well. The jar version of commons-httpclient is 3.1.
I got a thread dump of the process, and one thread seems to be waiting
on
the org.apache.commons.httpclient.MultiThreadedHttpConnectionManager as
shown below. All other threads are in a RUNNABLE state (besides the
Finalizer daemon).
[java] Full thread dump Java HotSpot(TM) 64-Bit Server VM (16.3-b01
mixed mode):
[java]
[java] "MultiThreadedHttpConnectionManager cleanup" daemon prio=10
tid=0x00007f441051c800 nid=0x527c in Object.wait() [0x00007f4417e2f000]
[java] java.lang.Thread.State: WAITING (on object monitor)
[java] at java.lang.Object.wait(Native Method)
[java] - waiting on <0x00007f443ae5b290> (a
java.lang.ref.ReferenceQueue$Lock)
[java] at
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
[java] - locked <0x00007f443ae5b290> (a
java.lang.ref.ReferenceQueue$Lock)
[java] at
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
[java] at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$Referen
ceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122)
Any ideas?
Thanks.