You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Dadasheva, Olga" <ol...@harvard.edu> on 2009/09/17 03:34:13 UTC
Latest trunk locks execution thread in SolrCore.getSearcher()
Hi,
I am testing EmbeddedSolrServer vs StreamingUpdateSolrServer for my
crawlers using more or less recent Solr code and everything was fine
till today when I took the latest trunk code.
When I start my crawler I see a number of INFO outputs
2009-09-16 21:08:29,399 INFO Adding
component:org.apache.solr.handler.component.HighlightComponent@36ae83
(SearchHandler.java:132) - [main]
2009-09-16 21:08:29,400 INFO Adding
component:org.apache.solr.handler.component.StatsComponent@1fb24d3
(SearchHandler.java:132) - [main]
2009-09-16 21:08:29,401 INFO Adding
component:org.apache.solr.handler.component.TermVectorComponent@14ba9a2
(SearchHandler.java:132) - [main]
2009-09-16 21:08:29,402 INFO Adding debug
component:org.apache.solr.handler.component.DebugComponent@12ea1dd
(SearchHandler.java:137) - [main]
and then the log/program stops.
The thread dump reveals the following:
"main" prio=3 tid=0x00030000 nid=0x2 in Object.wait()
[0xfe67c000..0xfe67fd80]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xeaaf6b10> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:991)
- locked <0xeaaf6b10> (a java.lang.Object)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:904)
at
org.apache.solr.handler.ReplicationHandler.getIndexVersion(ReplicationHa
ndler.java:472)
at
org.apache.solr.handler.ReplicationHandler.getStatistics(ReplicationHand
ler.java:490)
at
org.apache.solr.core.JmxMonitoredMap$SolrDynamicMBean.getMBeanInfo(JmxMo
nitoredMap.java:224)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassNa
me(DefaultMBeanServerInterceptor.java:321)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(Defa
ultMBeanServerInterceptor.java:307)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java
:482)
at
org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:137)
at
org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:47)
at
org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:4
46)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:578)
at
harvard.solr.search.service.EmbeddedSearchService.setSolrHome(EmbeddedSe
archService.java:47)
The same is happening for the StreamingUpdateSolrServer.
Do you think it's a bug?
Thank you for looking into it,
-Olga
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Chris Harris <ry...@gmail.com>.
Solr now starts up fine in Tomcat with both JMX and ReplicationHandler
enabled if I construct my working copy as follows:
download solr-r815830
reverse merge r815587 ("SOLR-1427: fixed registry MBean issue")
apply 1427.afterlatch.patch
I haven't tried applying 1427.afterlatch.patch to the svn head, but I
assume the results would be the same.
2009/9/20 Chris Hostetter <ho...@fucit.org>:
>
> : instance, which is based on Solr SVN r815830. This patch did not seem
> : to solve the hang problem; once I reenabled JMX, then the process
> : would hang at the same spot, i.e. right after
>
> Yeah, I believe the "hang" is really orthoginal to the SearchComponents
> issue and seems to be because of the types of Statistics
> ReplicationHandler wants to expose and the way certain JMX implementaions
> deal with a newly registered MBean -- the changes made in SOLR-1427 (which
> yonik has since reverted) just exposed this problem by registering hte
> MBeans at an inopportune time.
>
> I'd apprecaite it if anyone who observed this problem in their
> version of Java could try out this patch...
>
> https://issues.apache.org/jira/secure/attachment/12420151/SOLR-1427.afterlatch.patch
>
> ...with both the ReplicationHandler and JMX turned on. The patch can be
> applied to the current trunk (where yonik reverted the previous code that
> caused the hang)
>
> If things still lock up even with that patch, a full threaddump would be
> much apprecaited.
>
>
> -Hoss
>
>
RE: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by "Dadasheva, Olga" <ol...@harvard.edu>.
HURAH! Works for me!
Thank you!
-----Original Message-----
From: Chris Hostetter [mailto:hossman_lucene@fucit.org]
Sent: Sunday, September 20, 2009 6:49 PM
To: solr-user@lucene.apache.org
Subject: Re: Latest trunk locks execution thread in
SolrCore.getSearcher()
: instance, which is based on Solr SVN r815830. This patch did not seem
: to solve the hang problem; once I reenabled JMX, then the process
: would hang at the same spot, i.e. right after
Yeah, I believe the "hang" is really orthoginal to the SearchComponents
issue and seems to be because of the types of Statistics
ReplicationHandler wants to expose and the way certain JMX
implementaions deal with a newly registered MBean -- the changes made in
SOLR-1427 (which yonik has since reverted) just exposed this problem by
registering hte MBeans at an inopportune time.
I'd apprecaite it if anyone who observed this problem in their version
of Java could try out this patch...
https://issues.apache.org/jira/secure/attachment/12420151/SOLR-1427.afte
rlatch.patch
...with both the ReplicationHandler and JMX turned on. The patch can be
applied to the current trunk (where yonik reverted the previous code
that caused the hang)
If things still lock up even with that patch, a full threaddump would be
much apprecaited.
-Hoss
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Chris Hostetter <ho...@fucit.org>.
: instance, which is based on Solr SVN r815830. This patch did not seem
: to solve the hang problem; once I reenabled JMX, then the process
: would hang at the same spot, i.e. right after
Yeah, I believe the "hang" is really orthoginal to the SearchComponents
issue and seems to be because of the types of Statistics
ReplicationHandler wants to expose and the way certain JMX implementaions
deal with a newly registered MBean -- the changes made in SOLR-1427 (which
yonik has since reverted) just exposed this problem by registering hte
MBeans at an inopportune time.
I'd apprecaite it if anyone who observed this problem in their
version of Java could try out this patch...
https://issues.apache.org/jira/secure/attachment/12420151/SOLR-1427.afterlatch.patch
...with both the ReplicationHandler and JMX turned on. The patch can be
applied to the current trunk (where yonik reverted the previous code that
caused the hang)
If things still lock up even with that patch, a full threaddump would be
much apprecaited.
-Hoss
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Chris Harris <ry...@gmail.com>.
No, I'm pretty sure nothing implements SolrInfoMBean.
I applied the new 1K version of SOLR-1427.patch from
https://issues.apache.org/jira/browse/SOLR-1427
(which appears to be a secondary patch, to be applied once the main
SOLR-1427 patch has already been applied) to my problematic Solr
instance, which is based on Solr SVN r815830. This patch did not seem
to solve the hang problem; once I reenabled JMX, then the process
would hang at the same spot, i.e. right after
INFO: Adding debug
component:org.apache.solr.handler.component.DebugComponent@1d7b222
appeared in the Tomcat log.
When Solr/Tomcat are hung, there are two Solr-related threads that
show up in a thread dump. I'll paste those stack traces below:
"pool-1-thread-1" prio=6 tid=0x0b1ef800 nid=0xdc8 waiting on condition
[0x0b68f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x035f3e60> (a
java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(Unknown Source)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown
Source)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(Unknown
Source)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(Unknown
Source)
at java.util.concurrent.CountDownLatch.await(Unknown Source)
at org.apache.solr.core.SolrCore$1.call(SolrCore.java:559)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
"Thread-1" prio=6 tid=0x00c92c00 nid=0xf14 in Object.wait() [0x0b19e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x035d2ab0> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:994)
- locked <0x035d2ab0> (a java.lang.Object)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:907)
at org.apache.solr.handler.ReplicationHandler.getIndexVersion(ReplicationHandler.java:472)
at org.apache.solr.handler.ReplicationHandler.getStatistics(ReplicationHandler.java:490)
at org.apache.solr.core.JmxMonitoredMap$SolrDynamicMBean.getMBeanInfo(JmxMonitoredMap.java:224)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassName(Unknown
Source)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(Unknown
Source)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(Unknown Source)
at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:137)
at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:47)
at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:446)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:578)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:425)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:278)
at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83)
at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275)
<snip>
2009/9/18 Grant Ingersoll <gs...@apache.org>:
> Also, do you have any custom components or anything that implements
> SolrInfoMBean?
>
> On Sep 18, 2009, at 8:16 AM, Grant Ingersoll wrote:
>
>> Can you try the patch I just put up on
>> https://issues.apache.org/jira/browse/SOLR-1427 and let me know if it works
>> when JMX is enabled?
>>
>> Also, do you have warming queries setup?
>>
>> On Sep 17, 2009, at 12:46 PM, Chris Harris wrote:
>>
>>> It looks like this works as a fix for me as well. (I'm not currently
>>> using JMX for anything anyway.)
>>>
>>> Curiously, the single-core example solrconfig.xml also has "<jmx />",
>>> but it doesn't seem to be a problem there.
>>>
>>> 2009/9/17 Dadasheva, Olga <ol...@harvard.edu>:
>>>>
>>>> Hi,
>>>>
>>>> FWIW: disabling <jmx/> fixed this problem for me.
>>>>
>>>> Thanks you!
>>>>
>>>> -Olga
>>>>
>>>> -----Original Message-----
>>>> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik
>>>> Seeley
>>>> Sent: Thursday, September 17, 2009 1:09 PM
>>>> To: solr-user@lucene.apache.org
>>>> Subject: Re: Latest trunk locks execution thread in
>>>> SolrCore.getSearcher()
>>>>
>>>> Interesting... I still haven't been able to reproduce a hang with either
>>>> jetty or tomcat.
>>>> I enabled replication and JMX... still nothing.
>>>>
>>>> -Yonik
>>>> http://www.lucidimagination.com
>>>>
>>>>
>>>> On Thu, Sep 17, 2009 at 12:35 PM, Chris Harris <ry...@gmail.com>
>>>> wrote:
>>>>>
>>>>> I found what looks like the same issue when I tried to install r815830
>>>>> under Tomcat. (It works ok with the normal Jetty example/start.jar.) I
>>>>> haven't checked the stack trace, but Tomcat would hang right after the
>>>>> message
>>>>>
>>>>> INFO: Adding debug
>>>>> component:org.apache.solr.handler.component.DebugComponent@1904e0d
>>>>>
>>>>> showed up in the log.
>>>>>
>>>>> I have a little more evidence about Yonik's theory that SOLR-1427 is
>>>>> part of the cause. In particular, when I reverse-merged r815587 (the
>>>>> commit for SOLR-1427) into (out of?) my r815830-based working copy,
>>>>> then Tomcat was able to load Solr normally.
>>>>>
>>>>> 2009/9/16 Yonik Seeley <yo...@lucidimagination.com>:
>>>>>>
>>>>>> On a quick look, it looks like this was caused (or at least triggered
>>>>>> by)
>>>>>> https://issues.apache.org/jira/browse/SOLR-1427
>>>>>>
>>>>>> Registering the bean in the SolrCore constructor causes it to
>>>>>> immediately turn around and ask for the stats which asks for a
>>>>>> searcher, which blocks.
>>>>>>
>>>>>> -Yonik
>>>>>> http://www.lucidimagination.com
>>>>>>
>>>>>> On Wed, Sep 16, 2009 at 9:34 PM, Dadasheva, Olga
>>>>>> <ol...@harvard.edu> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am testing EmbeddedSolrServer vs StreamingUpdateSolrServer for
>>>>>>> my crawlers using more or less recent Solr code and everything was
>>>>>>> fine till today when I took the latest trunk code.
>>>>>>> When I start my crawler I see a number of INFO outputs
>>>>>>> 2009-09-16 21:08:29,399 INFO Adding
>>>>>>> component:org.apache.solr.handler.component.HighlightComponent@36ae8
>>>>>>> 3
>>>>>>> (SearchHandler.java:132) - [main]
>>>>>>> 2009-09-16 21:08:29,400 INFO Adding
>>>>>>> component:org.apache.solr.handler.component.StatsComponent@1fb24d3
>>>>>>> (SearchHandler.java:132) - [main]
>>>>>>> 2009-09-16 21:08:29,401 INFO Adding
>>>>>>> component:org.apache.solr.handler.component.TermVectorComponent@14ba
>>>>>>> 9a2
>>>>>>> (SearchHandler.java:132) - [main]
>>>>>>> 2009-09-16 21:08:29,402 INFO Adding debug
>>>>>>> component:org.apache.solr.handler.component.DebugComponent@12ea1dd
>>>>>>> (SearchHandler.java:137) - [main]
>>>>>>>
>>>>>>> and then the log/program stops.
>>>>>
>>>>
>>
>> --------------------------
>> Grant Ingersoll
>> http://www.lucidimagination.com/
>>
>> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
>> Solr/Lucene:
>> http://www.lucidimagination.com/search
>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
> Solr/Lucene:
> http://www.lucidimagination.com/search
>
>
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Grant Ingersoll <gs...@apache.org>.
Also, do you have any custom components or anything that implements
SolrInfoMBean?
On Sep 18, 2009, at 8:16 AM, Grant Ingersoll wrote:
> Can you try the patch I just put up on https://issues.apache.org/jira/browse/SOLR-1427
> and let me know if it works when JMX is enabled?
>
> Also, do you have warming queries setup?
>
> On Sep 17, 2009, at 12:46 PM, Chris Harris wrote:
>
>> It looks like this works as a fix for me as well. (I'm not currently
>> using JMX for anything anyway.)
>>
>> Curiously, the single-core example solrconfig.xml also has "<jmx />",
>> but it doesn't seem to be a problem there.
>>
>> 2009/9/17 Dadasheva, Olga <ol...@harvard.edu>:
>>> Hi,
>>>
>>> FWIW: disabling <jmx/> fixed this problem for me.
>>>
>>> Thanks you!
>>>
>>> -Olga
>>>
>>> -----Original Message-----
>>> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of
>>> Yonik Seeley
>>> Sent: Thursday, September 17, 2009 1:09 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Latest trunk locks execution thread in
>>> SolrCore.getSearcher()
>>>
>>> Interesting... I still haven't been able to reproduce a hang with
>>> either jetty or tomcat.
>>> I enabled replication and JMX... still nothing.
>>>
>>> -Yonik
>>> http://www.lucidimagination.com
>>>
>>>
>>> On Thu, Sep 17, 2009 at 12:35 PM, Chris Harris <ry...@gmail.com>
>>> wrote:
>>>> I found what looks like the same issue when I tried to install
>>>> r815830
>>>> under Tomcat. (It works ok with the normal Jetty example/
>>>> start.jar.) I
>>>> haven't checked the stack trace, but Tomcat would hang right
>>>> after the
>>>> message
>>>>
>>>> INFO: Adding debug
>>>> component:org.apache.solr.handler.component.DebugComponent@1904e0d
>>>>
>>>> showed up in the log.
>>>>
>>>> I have a little more evidence about Yonik's theory that SOLR-1427
>>>> is
>>>> part of the cause. In particular, when I reverse-merged r815587
>>>> (the
>>>> commit for SOLR-1427) into (out of?) my r815830-based working copy,
>>>> then Tomcat was able to load Solr normally.
>>>>
>>>> 2009/9/16 Yonik Seeley <yo...@lucidimagination.com>:
>>>>> On a quick look, it looks like this was caused (or at least
>>>>> triggered
>>>>> by)
>>>>> https://issues.apache.org/jira/browse/SOLR-1427
>>>>>
>>>>> Registering the bean in the SolrCore constructor causes it to
>>>>> immediately turn around and ask for the stats which asks for a
>>>>> searcher, which blocks.
>>>>>
>>>>> -Yonik
>>>>> http://www.lucidimagination.com
>>>>>
>>>>> On Wed, Sep 16, 2009 at 9:34 PM, Dadasheva, Olga
>>>>> <ol...@harvard.edu> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I am testing EmbeddedSolrServer vs StreamingUpdateSolrServer
>>>>>> for
>>>>>> my crawlers using more or less recent Solr code and everything
>>>>>> was
>>>>>> fine till today when I took the latest trunk code.
>>>>>> When I start my crawler I see a number of INFO outputs
>>>>>> 2009-09-16 21:08:29,399 INFO Adding
>>>>>> component:org.apache.solr.handler.component.HighlightComponent
>>>>>> @36ae8
>>>>>> 3
>>>>>> (SearchHandler.java:132) - [main]
>>>>>> 2009-09-16 21:08:29,400 INFO Adding
>>>>>> component:org.apache.solr.handler.component.StatsComponent
>>>>>> @1fb24d3
>>>>>> (SearchHandler.java:132) - [main]
>>>>>> 2009-09-16 21:08:29,401 INFO Adding
>>>>>> component:org.apache.solr.handler.component.TermVectorComponent
>>>>>> @14ba
>>>>>> 9a2
>>>>>> (SearchHandler.java:132) - [main]
>>>>>> 2009-09-16 21:08:29,402 INFO Adding debug
>>>>>> component:org.apache.solr.handler.component.DebugComponent
>>>>>> @12ea1dd
>>>>>> (SearchHandler.java:137) - [main]
>>>>>>
>>>>>> and then the log/program stops.
>>>>
>>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
> using Solr/Lucene:
> http://www.lucidimagination.com/search
>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Chris Harris <ry...@gmail.com>.
Forgot to answer this one. Yes, I do have a warming query to get the
sort caches up to speed. I think it takes a while to run; my guess
would be 30 seconds or so.
2009/9/18 Grant Ingersoll <gs...@apache.org>:
>
> Also, do you have warming queries setup?
>
> On Sep 17, 2009, at 12:46 PM, Chris Harris wrote:
>
>> It looks like this works as a fix for me as well. (I'm not currently
>> using JMX for anything anyway.)
>>
>> Curiously, the single-core example solrconfig.xml also has "<jmx />",
>> but it doesn't seem to be a problem there.
>>
>> 2009/9/17 Dadasheva, Olga <ol...@harvard.edu>:
>>>
>>> Hi,
>>>
>>> FWIW: disabling <jmx/> fixed this problem for me.
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Grant Ingersoll <gs...@apache.org>.
Can you try the patch I just put up on https://issues.apache.org/jira/browse/SOLR-1427
and let me know if it works when JMX is enabled?
Also, do you have warming queries setup?
On Sep 17, 2009, at 12:46 PM, Chris Harris wrote:
> It looks like this works as a fix for me as well. (I'm not currently
> using JMX for anything anyway.)
>
> Curiously, the single-core example solrconfig.xml also has "<jmx />",
> but it doesn't seem to be a problem there.
>
> 2009/9/17 Dadasheva, Olga <ol...@harvard.edu>:
>> Hi,
>>
>> FWIW: disabling <jmx/> fixed this problem for me.
>>
>> Thanks you!
>>
>> -Olga
>>
>> -----Original Message-----
>> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of
>> Yonik Seeley
>> Sent: Thursday, September 17, 2009 1:09 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Latest trunk locks execution thread in
>> SolrCore.getSearcher()
>>
>> Interesting... I still haven't been able to reproduce a hang with
>> either jetty or tomcat.
>> I enabled replication and JMX... still nothing.
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>>
>> On Thu, Sep 17, 2009 at 12:35 PM, Chris Harris <ry...@gmail.com>
>> wrote:
>>> I found what looks like the same issue when I tried to install
>>> r815830
>>> under Tomcat. (It works ok with the normal Jetty example/
>>> start.jar.) I
>>> haven't checked the stack trace, but Tomcat would hang right after
>>> the
>>> message
>>>
>>> INFO: Adding debug
>>> component:org.apache.solr.handler.component.DebugComponent@1904e0d
>>>
>>> showed up in the log.
>>>
>>> I have a little more evidence about Yonik's theory that SOLR-1427 is
>>> part of the cause. In particular, when I reverse-merged r815587 (the
>>> commit for SOLR-1427) into (out of?) my r815830-based working copy,
>>> then Tomcat was able to load Solr normally.
>>>
>>> 2009/9/16 Yonik Seeley <yo...@lucidimagination.com>:
>>>> On a quick look, it looks like this was caused (or at least
>>>> triggered
>>>> by)
>>>> https://issues.apache.org/jira/browse/SOLR-1427
>>>>
>>>> Registering the bean in the SolrCore constructor causes it to
>>>> immediately turn around and ask for the stats which asks for a
>>>> searcher, which blocks.
>>>>
>>>> -Yonik
>>>> http://www.lucidimagination.com
>>>>
>>>> On Wed, Sep 16, 2009 at 9:34 PM, Dadasheva, Olga
>>>> <ol...@harvard.edu> wrote:
>>>>> Hi,
>>>>>
>>>>> I am testing EmbeddedSolrServer vs StreamingUpdateSolrServer for
>>>>> my crawlers using more or less recent Solr code and everything was
>>>>> fine till today when I took the latest trunk code.
>>>>> When I start my crawler I see a number of INFO outputs
>>>>> 2009-09-16 21:08:29,399 INFO Adding
>>>>> component:org.apache.solr.handler.component.HighlightComponent
>>>>> @36ae8
>>>>> 3
>>>>> (SearchHandler.java:132) - [main]
>>>>> 2009-09-16 21:08:29,400 INFO Adding
>>>>> component:org.apache.solr.handler.component.StatsComponent@1fb24d3
>>>>> (SearchHandler.java:132) - [main]
>>>>> 2009-09-16 21:08:29,401 INFO Adding
>>>>> component:org.apache.solr.handler.component.TermVectorComponent
>>>>> @14ba
>>>>> 9a2
>>>>> (SearchHandler.java:132) - [main]
>>>>> 2009-09-16 21:08:29,402 INFO Adding debug
>>>>> component:org.apache.solr.handler.component.DebugComponent@12ea1dd
>>>>> (SearchHandler.java:137) - [main]
>>>>>
>>>>> and then the log/program stops.
>>>
>>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Chris Harris <ry...@gmail.com>.
It looks like this works as a fix for me as well. (I'm not currently
using JMX for anything anyway.)
Curiously, the single-core example solrconfig.xml also has "<jmx />",
but it doesn't seem to be a problem there.
2009/9/17 Dadasheva, Olga <ol...@harvard.edu>:
> Hi,
>
> FWIW: disabling <jmx/> fixed this problem for me.
>
> Thanks you!
>
> -Olga
>
> -----Original Message-----
> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
> Sent: Thursday, September 17, 2009 1:09 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Latest trunk locks execution thread in SolrCore.getSearcher()
>
> Interesting... I still haven't been able to reproduce a hang with either jetty or tomcat.
> I enabled replication and JMX... still nothing.
>
> -Yonik
> http://www.lucidimagination.com
>
>
> On Thu, Sep 17, 2009 at 12:35 PM, Chris Harris <ry...@gmail.com> wrote:
>> I found what looks like the same issue when I tried to install r815830
>> under Tomcat. (It works ok with the normal Jetty example/start.jar.) I
>> haven't checked the stack trace, but Tomcat would hang right after the
>> message
>>
>> INFO: Adding debug
>> component:org.apache.solr.handler.component.DebugComponent@1904e0d
>>
>> showed up in the log.
>>
>> I have a little more evidence about Yonik's theory that SOLR-1427 is
>> part of the cause. In particular, when I reverse-merged r815587 (the
>> commit for SOLR-1427) into (out of?) my r815830-based working copy,
>> then Tomcat was able to load Solr normally.
>>
>> 2009/9/16 Yonik Seeley <yo...@lucidimagination.com>:
>>> On a quick look, it looks like this was caused (or at least triggered
>>> by)
>>> https://issues.apache.org/jira/browse/SOLR-1427
>>>
>>> Registering the bean in the SolrCore constructor causes it to
>>> immediately turn around and ask for the stats which asks for a
>>> searcher, which blocks.
>>>
>>> -Yonik
>>> http://www.lucidimagination.com
>>>
>>> On Wed, Sep 16, 2009 at 9:34 PM, Dadasheva, Olga
>>> <ol...@harvard.edu> wrote:
>>>> Hi,
>>>>
>>>> I am testing EmbeddedSolrServer vs StreamingUpdateSolrServer for
>>>> my crawlers using more or less recent Solr code and everything was
>>>> fine till today when I took the latest trunk code.
>>>> When I start my crawler I see a number of INFO outputs
>>>> 2009-09-16 21:08:29,399 INFO Adding
>>>> component:org.apache.solr.handler.component.HighlightComponent@36ae8
>>>> 3
>>>> (SearchHandler.java:132) - [main]
>>>> 2009-09-16 21:08:29,400 INFO Adding
>>>> component:org.apache.solr.handler.component.StatsComponent@1fb24d3
>>>> (SearchHandler.java:132) - [main]
>>>> 2009-09-16 21:08:29,401 INFO Adding
>>>> component:org.apache.solr.handler.component.TermVectorComponent@14ba
>>>> 9a2
>>>> (SearchHandler.java:132) - [main]
>>>> 2009-09-16 21:08:29,402 INFO Adding debug
>>>> component:org.apache.solr.handler.component.DebugComponent@12ea1dd
>>>> (SearchHandler.java:137) - [main]
>>>>
>>>> and then the log/program stops.
>>
>
RE: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by "Dadasheva, Olga" <ol...@harvard.edu>.
Hi,
FWIW: disabling <jmx/> fixed this problem for me.
Thanks you!
-Olga
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Thursday, September 17, 2009 1:09 PM
To: solr-user@lucene.apache.org
Subject: Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Interesting... I still haven't been able to reproduce a hang with either jetty or tomcat.
I enabled replication and JMX... still nothing.
-Yonik
http://www.lucidimagination.com
On Thu, Sep 17, 2009 at 12:35 PM, Chris Harris <ry...@gmail.com> wrote:
> I found what looks like the same issue when I tried to install r815830
> under Tomcat. (It works ok with the normal Jetty example/start.jar.) I
> haven't checked the stack trace, but Tomcat would hang right after the
> message
>
> INFO: Adding debug
> component:org.apache.solr.handler.component.DebugComponent@1904e0d
>
> showed up in the log.
>
> I have a little more evidence about Yonik's theory that SOLR-1427 is
> part of the cause. In particular, when I reverse-merged r815587 (the
> commit for SOLR-1427) into (out of?) my r815830-based working copy,
> then Tomcat was able to load Solr normally.
>
> 2009/9/16 Yonik Seeley <yo...@lucidimagination.com>:
>> On a quick look, it looks like this was caused (or at least triggered
>> by)
>> https://issues.apache.org/jira/browse/SOLR-1427
>>
>> Registering the bean in the SolrCore constructor causes it to
>> immediately turn around and ask for the stats which asks for a
>> searcher, which blocks.
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>> On Wed, Sep 16, 2009 at 9:34 PM, Dadasheva, Olga
>> <ol...@harvard.edu> wrote:
>>> Hi,
>>>
>>> I am testing EmbeddedSolrServer vs StreamingUpdateSolrServer for
>>> my crawlers using more or less recent Solr code and everything was
>>> fine till today when I took the latest trunk code.
>>> When I start my crawler I see a number of INFO outputs
>>> 2009-09-16 21:08:29,399 INFO Adding
>>> component:org.apache.solr.handler.component.HighlightComponent@36ae8
>>> 3
>>> (SearchHandler.java:132) - [main]
>>> 2009-09-16 21:08:29,400 INFO Adding
>>> component:org.apache.solr.handler.component.StatsComponent@1fb24d3
>>> (SearchHandler.java:132) - [main]
>>> 2009-09-16 21:08:29,401 INFO Adding
>>> component:org.apache.solr.handler.component.TermVectorComponent@14ba
>>> 9a2
>>> (SearchHandler.java:132) - [main]
>>> 2009-09-16 21:08:29,402 INFO Adding debug
>>> component:org.apache.solr.handler.component.DebugComponent@12ea1dd
>>> (SearchHandler.java:137) - [main]
>>>
>>> and then the log/program stops.
>
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Chris Hostetter <ho...@fucit.org>.
: Maybe moving the resourceLoader.inform(infoRegistry) call PRIOR to
: resourceLoader.inform( this ) *or* AFTER latch.countDown() would solve
: this problem?
If this really is the problem, then a more general purpose solution to
future proof us against similar problems down the road would probably be
to get rid of the current "if (config.jmxConfig.enabled)" logic for
initializing SolrCore.infoRegistry, and instead us something like...
infoRegistry = new ConcurrentHashMap<String, SolrInfoMBean>();
...
// do all initialization, add things to infoRegistry as needed
...
// done with all initilation, core and all MBeans are fully functional
if (config.jmxConfig.enabled) {
Map tmp = infoRegistry;
infoRegistry = new JmxMonitoredMap<String, SolrInfoMBean>(name, config.jmxConfig);
infoRegistry.putAll(tmp);
} else {
log.info("JMX monitoring not detected for core: " + name);
}
...that way we wouldn't have to worry about any other type of resource
contention that might happen from the JMX monitor wanting ot get info from
the MBeans as soon as they are added to the registry.
-Hoss
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Chris Hostetter <ho...@fucit.org>.
: Interesting... I still haven't been able to reproduce a hang with
: either jetty or tomcat.
: I enabled replication and JMX... still nothing.
I haven't tried to reproduce the problem, and i don't even have a concrete
theory as to what the problem is ... but i did want to point out something
that smells fishy...
looking over the patch in SOLR-1427, i notice that the patch adds new code
so that SolrResourceLoader adds everything it instantiates to the registry
map -- but there are no lines in that patch removing any code that was
already adding stuff to the infoRegistry ... which means any bean
types that were already getting instantiated by the ResourceLoader, and explicitly
added to hte registry are probably now getting added to the registry twice
... for a normal Map that's not a big deal, but perhaps the JMX backed map
has some problems with that? (it's not likely something that we ever
tested extensively)
Alternately: maybe the problem isn't with adding to the JMX backed map
twice ... maybe the issue is what state the beans are in each time we add
them to the map ... the first time requesthandlers are added to the
infoRegistry it's prior to the inform(SolrCore) call so they havne't been
fully initalized yet -- but the way SOlrResourceLoader adds them now (the
second time) is *after* inform(SolrCore) has been called and in between
the creation/countDown of the CountDownLatch for the searchExecutor ...
...hmmm, actually this starting to sound like a concrete theory,
ReplicationHandler.getStatistics has very different behavior before/after
inform(SolrCore) has been called....
Maybe moving the resourceLoader.inform(infoRegistry) call PRIOR to
resourceLoader.inform( this ) *or* AFTER latch.countDown() would solve
this problem?
-Hoss
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Yonik Seeley <yo...@lucidimagination.com>.
Interesting... I still haven't been able to reproduce a hang with
either jetty or tomcat.
I enabled replication and JMX... still nothing.
-Yonik
http://www.lucidimagination.com
On Thu, Sep 17, 2009 at 12:35 PM, Chris Harris <ry...@gmail.com> wrote:
> I found what looks like the same issue when I tried to install r815830
> under Tomcat. (It works ok with the normal Jetty example/start.jar.) I
> haven't checked the stack trace, but Tomcat would hang right after the
> message
>
> INFO: Adding debug
> component:org.apache.solr.handler.component.DebugComponent@1904e0d
>
> showed up in the log.
>
> I have a little more evidence about Yonik's theory that SOLR-1427 is
> part of the cause. In particular, when I reverse-merged r815587 (the
> commit for SOLR-1427) into (out of?) my r815830-based working copy,
> then Tomcat was able to load Solr normally.
>
> 2009/9/16 Yonik Seeley <yo...@lucidimagination.com>:
>> On a quick look, it looks like this was caused (or at least triggered by)
>> https://issues.apache.org/jira/browse/SOLR-1427
>>
>> Registering the bean in the SolrCore constructor causes it to
>> immediately turn around and ask for the stats which asks for a
>> searcher, which blocks.
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>> On Wed, Sep 16, 2009 at 9:34 PM, Dadasheva, Olga
>> <ol...@harvard.edu> wrote:
>>> Hi,
>>>
>>> I am testing EmbeddedSolrServer vs StreamingUpdateSolrServer for my
>>> crawlers using more or less recent Solr code and everything was fine
>>> till today when I took the latest trunk code.
>>> When I start my crawler I see a number of INFO outputs
>>> 2009-09-16 21:08:29,399 INFO Adding
>>> component:org.apache.solr.handler.component.HighlightComponent@36ae83
>>> (SearchHandler.java:132) - [main]
>>> 2009-09-16 21:08:29,400 INFO Adding
>>> component:org.apache.solr.handler.component.StatsComponent@1fb24d3
>>> (SearchHandler.java:132) - [main]
>>> 2009-09-16 21:08:29,401 INFO Adding
>>> component:org.apache.solr.handler.component.TermVectorComponent@14ba9a2
>>> (SearchHandler.java:132) - [main]
>>> 2009-09-16 21:08:29,402 INFO Adding debug
>>> component:org.apache.solr.handler.component.DebugComponent@12ea1dd
>>> (SearchHandler.java:137) - [main]
>>>
>>> and then the log/program stops.
>
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Chris Harris <ry...@gmail.com>.
Ok, let me revise my previous report:
It turns out if I take the single-core or multicore example stuff from
running "ant example" on r815830 and plop them into Tomcat, this
doesn't hang. However, if I plop Solr as I configure it for production
use into Tomcat, it does hang.
The main ways I can think of how my Solr conf differs from the example
configs are the use of ExtractingRequestHandler and of the Solr qsol
patch (https://issues.apache.org/jira/browse/SOLR-896). As an
experiment, I tried removing apache-solr-cell-1.4-dev.jar and
apache-solr-qsol-1.4-dev.jar from my Solr lib directory, and removing
reference to qsol from my solrconfig.xml, but this does not prevent
Solr from hanging in Tomcat; in this case Tomcat hung in the same spot
as I reported before.
I did find one way to get my production-like Solr configuration not to
hang on startup. That was to 1) remove apache-solr-qsol-1.4-dev.jar
from my lib/ directory, but to 2) still leave this config option in my
solrconfig.xml:
<queryParser name="qsol" class="org.apache.solr.search.QsolQParserPlugin"/>
Something about the class loading exception seems to get around the bean issue.
Here's an excerpt from my Tomcat log:
...
Sep 17, 2009 12:01:22 PM org.apache.solr.core.SolrCore <init>
INFO: [filingcore] Opening new SolrCore at
c:\tomcat-solr\solrme\filingcore\,
dataDir=c:\tomcat-solr\solrme\filingcore\data\
Sep 17, 2009 12:01:22 PM org.apache.solr.core.JmxMonitoredMap <init>
INFO: JMX monitoring is enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@ecb67f
Sep 17, 2009 12:01:22 PM org.apache.solr.core.SolrCore initListeners
INFO: [filingcore] Added SolrEventListener:
org.apache.solr.core.QuerySenderListener{queries=[{q=*:*,sort=date
asc,companynameforsorting asc,formtype asc,start=0,rows=10}]}
Sep 17, 2009 12:01:22 PM org.apache.solr.core.SolrCore initListeners
INFO: [filingcore] Added SolrEventListener:
org.apache.solr.core.QuerySenderListener{queries=[{q=*:*,sort=date
asc,companynameforsorting asc,formtype asc,start=0,rows=10}]}
Sep 17, 2009 12:01:22 PM org.apache.solr.request.XSLTResponseWriter init
INFO: xsltCacheLifetimeSeconds=5
Sep 17, 2009 12:01:22 PM org.apache.solr.core.SolrCore finalize
SEVERE: REFCOUNT ERROR: unreferenced
org.apache.solr.core.SolrCore@4eeaaf (filingcore) has a reference
count of 1
Sep 17, 2009 12:01:22 PM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: Error loading class
'org.apache.solr.search.QsolQParserPlugin'
at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:311)
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:411)
at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1479)
at org.apache.solr.core.SolrCore.initQParsers(SolrCore.java:1433)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:533)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:425)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:278)
at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83)
at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275)
at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:397)
at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:108)
at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:3800)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4450)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:630)
at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:556)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:491)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1206)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:314)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at org.apache.catalina.core.StandardService.start(StandardService.java:516)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
at org.apache.catalina.startup.Catalina.start(Catalina.java:583)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: java.lang.ClassNotFoundException:
org.apache.solr.search.QsolQParserPlugin
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.net.FactoryURLClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClassInternal(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:295)
... 35 more
Sep 17, 2009 12:01:22 PM org.apache.solr.servlet.SolrDispatchFilter init
INFO: user.dir=C:\tomcat
Sep 17, 2009 12:01:22 PM org.apache.solr.servlet.SolrDispatchFilter init
INFO: SolrDispatchFilter.init() done
Sep 17, 2009 12:01:22 PM org.apache.solr.servlet.SolrServlet init
INFO: SolrServlet.init()
Sep 17, 2009 12:01:22 PM org.apache.solr.core.SolrResourceLoader locateSolrHome
INFO: Using JNDI solr.home: c:\tomcat-solr\solrme
Sep 17, 2009 12:01:22 PM org.apache.solr.servlet.SolrServlet init
INFO: SolrServlet.init() done
Sep 17, 2009 12:01:22 PM org.apache.solr.core.SolrResourceLoader locateSolrHome
INFO: Using JNDI solr.home: c:\tomcat-solr\solrme
Sep 17, 2009 12:01:22 PM org.apache.solr.servlet.SolrUpdateServlet init
INFO: SolrUpdateServlet.init() done
Sep 17, 2009 12:01:22 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-9000
Sep 17, 2009 12:01:22 PM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:8009
Sep 17, 2009 12:01:22 PM org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/32 config=null
Sep 17, 2009 12:01:22 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 2709 ms
2009/9/17 Chris Harris <ry...@gmail.com>:
> I found what looks like the same issue when I tried to install r815830
> under Tomcat. (It works ok with the normal Jetty example/start.jar.) I
> haven't checked the stack trace, but Tomcat would hang right after the
> message
>
> INFO: Adding debug
> component:org.apache.solr.handler.component.DebugComponent@1904e0d
>
> showed up in the log.
>
> I have a little more evidence about Yonik's theory that SOLR-1427 is
> part of the cause. In particular, when I reverse-merged r815587 (the
> commit for SOLR-1427) into (out of?) my r815830-based working copy,
> then Tomcat was able to load Solr normally.
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Chris Harris <ry...@gmail.com>.
I found what looks like the same issue when I tried to install r815830
under Tomcat. (It works ok with the normal Jetty example/start.jar.) I
haven't checked the stack trace, but Tomcat would hang right after the
message
INFO: Adding debug
component:org.apache.solr.handler.component.DebugComponent@1904e0d
showed up in the log.
I have a little more evidence about Yonik's theory that SOLR-1427 is
part of the cause. In particular, when I reverse-merged r815587 (the
commit for SOLR-1427) into (out of?) my r815830-based working copy,
then Tomcat was able to load Solr normally.
2009/9/16 Yonik Seeley <yo...@lucidimagination.com>:
> On a quick look, it looks like this was caused (or at least triggered by)
> https://issues.apache.org/jira/browse/SOLR-1427
>
> Registering the bean in the SolrCore constructor causes it to
> immediately turn around and ask for the stats which asks for a
> searcher, which blocks.
>
> -Yonik
> http://www.lucidimagination.com
>
> On Wed, Sep 16, 2009 at 9:34 PM, Dadasheva, Olga
> <ol...@harvard.edu> wrote:
>> Hi,
>>
>> I am testing EmbeddedSolrServer vs StreamingUpdateSolrServer for my
>> crawlers using more or less recent Solr code and everything was fine
>> till today when I took the latest trunk code.
>> When I start my crawler I see a number of INFO outputs
>> 2009-09-16 21:08:29,399 INFO Adding
>> component:org.apache.solr.handler.component.HighlightComponent@36ae83
>> (SearchHandler.java:132) - [main]
>> 2009-09-16 21:08:29,400 INFO Adding
>> component:org.apache.solr.handler.component.StatsComponent@1fb24d3
>> (SearchHandler.java:132) - [main]
>> 2009-09-16 21:08:29,401 INFO Adding
>> component:org.apache.solr.handler.component.TermVectorComponent@14ba9a2
>> (SearchHandler.java:132) - [main]
>> 2009-09-16 21:08:29,402 INFO Adding debug
>> component:org.apache.solr.handler.component.DebugComponent@12ea1dd
>> (SearchHandler.java:137) - [main]
>>
>> and then the log/program stops.
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Grant Ingersoll <gs...@apache.org>.
On Sep 16, 2009, at 6:54 PM, Yonik Seeley wrote:
> On a quick look, it looks like this was caused (or at least
> triggered by)
> https://issues.apache.org/jira/browse/SOLR-1427
>
> Registering the bean in the SolrCore constructor causes it to
> immediately turn around and ask for the stats which asks for a
> searcher, which blocks.
Hmm, yeah that sounds likely. We can revert if need be and push to 1.5
>
> -Yonik
> http://www.lucidimagination.com
>
> On Wed, Sep 16, 2009 at 9:34 PM, Dadasheva, Olga
> <ol...@harvard.edu> wrote:
>> Hi,
>>
>> I am testing EmbeddedSolrServer vs StreamingUpdateSolrServer for my
>> crawlers using more or less recent Solr code and everything was fine
>> till today when I took the latest trunk code.
>> When I start my crawler I see a number of INFO outputs
>> 2009-09-16 21:08:29,399 INFO Adding
>> component:org.apache.solr.handler.component.HighlightComponent@36ae83
>> (SearchHandler.java:132) - [main]
>> 2009-09-16 21:08:29,400 INFO Adding
>> component:org.apache.solr.handler.component.StatsComponent@1fb24d3
>> (SearchHandler.java:132) - [main]
>> 2009-09-16 21:08:29,401 INFO Adding
>> component:org.apache.solr.handler.component.TermVectorComponent
>> @14ba9a2
>> (SearchHandler.java:132) - [main]
>> 2009-09-16 21:08:29,402 INFO Adding debug
>> component:org.apache.solr.handler.component.DebugComponent@12ea1dd
>> (SearchHandler.java:137) - [main]
>>
>> and then the log/program stops.
>>
>> The thread dump reveals the following:
>>
>> "main" prio=3 tid=0x00030000 nid=0x2 in Object.wait()
>> [0xfe67c000..0xfe67fd80]
>> java.lang.Thread.State: WAITING (on object monitor)
>> at java.lang.Object.wait(Native Method)
>> - waiting on <0xeaaf6b10> (a java.lang.Object)
>> at java.lang.Object.wait(Object.java:485)
>> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:
>> 991)
>> - locked <0xeaaf6b10> (a java.lang.Object)
>> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:
>> 904)
>> at
>> org.apache.solr.handler.ReplicationHandler.getIndexVersion
>> (ReplicationHa
>> ndler.java:472)
>> at
>> org.apache.solr.handler.ReplicationHandler.getStatistics
>> (ReplicationHand
>> ler.java:490)
>> at
>> org.apache.solr.core.JmxMonitoredMap$SolrDynamicMBean.getMBeanInfo
>> (JmxMo
>> nitoredMap.java:224)
>> at
>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassNa
>> me(DefaultMBeanServerInterceptor.java:321)
>> at
>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean
>> (Defa
>> ultMBeanServerInterceptor.java:307)
>> at
>> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean
>> (JmxMBeanServer.java
>> :482)
>> at
>> org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:137)
>> at
>> org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:47)
>> at
>> org.apache.solr.core.SolrResourceLoader.inform
>> (SolrResourceLoader.java:4
>> 46)
>> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:578)
>> at
>> harvard.solr.search.service.EmbeddedSearchService.setSolrHome
>> (EmbeddedSe
>> archService.java:47)
>>
>> The same is happening for the StreamingUpdateSolrServer.
>>
>> Do you think it's a bug?
>>
>> Thank you for looking into it,
>>
>> -Olga
>>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
Re: Latest trunk locks execution thread in SolrCore.getSearcher()
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On a quick look, it looks like this was caused (or at least triggered by)
https://issues.apache.org/jira/browse/SOLR-1427
Registering the bean in the SolrCore constructor causes it to
immediately turn around and ask for the stats which asks for a
searcher, which blocks.
-Yonik
http://www.lucidimagination.com
On Wed, Sep 16, 2009 at 9:34 PM, Dadasheva, Olga
<ol...@harvard.edu> wrote:
> Hi,
>
> I am testing EmbeddedSolrServer vs StreamingUpdateSolrServer for my
> crawlers using more or less recent Solr code and everything was fine
> till today when I took the latest trunk code.
> When I start my crawler I see a number of INFO outputs
> 2009-09-16 21:08:29,399 INFO Adding
> component:org.apache.solr.handler.component.HighlightComponent@36ae83
> (SearchHandler.java:132) - [main]
> 2009-09-16 21:08:29,400 INFO Adding
> component:org.apache.solr.handler.component.StatsComponent@1fb24d3
> (SearchHandler.java:132) - [main]
> 2009-09-16 21:08:29,401 INFO Adding
> component:org.apache.solr.handler.component.TermVectorComponent@14ba9a2
> (SearchHandler.java:132) - [main]
> 2009-09-16 21:08:29,402 INFO Adding debug
> component:org.apache.solr.handler.component.DebugComponent@12ea1dd
> (SearchHandler.java:137) - [main]
>
> and then the log/program stops.
>
> The thread dump reveals the following:
>
> "main" prio=3 tid=0x00030000 nid=0x2 in Object.wait()
> [0xfe67c000..0xfe67fd80]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xeaaf6b10> (a java.lang.Object)
> at java.lang.Object.wait(Object.java:485)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:991)
> - locked <0xeaaf6b10> (a java.lang.Object)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:904)
> at
> org.apache.solr.handler.ReplicationHandler.getIndexVersion(ReplicationHa
> ndler.java:472)
> at
> org.apache.solr.handler.ReplicationHandler.getStatistics(ReplicationHand
> ler.java:490)
> at
> org.apache.solr.core.JmxMonitoredMap$SolrDynamicMBean.getMBeanInfo(JmxMo
> nitoredMap.java:224)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassNa
> me(DefaultMBeanServerInterceptor.java:321)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(Defa
> ultMBeanServerInterceptor.java:307)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java
> :482)
> at
> org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:137)
> at
> org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:47)
> at
> org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:4
> 46)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:578)
> at
> harvard.solr.search.service.EmbeddedSearchService.setSolrHome(EmbeddedSe
> archService.java:47)
>
> The same is happening for the StreamingUpdateSolrServer.
>
> Do you think it's a bug?
>
> Thank you for looking into it,
>
> -Olga
>