Re: Too many connections in CLOSE_WAIT state on master solr server

[samarth s <sa...@gmail.com>]

Hi Ranveer,

You can try this '-Dhttp.maxConnections' out, may resolve the issue.
But the root cause I figured may lie with some queries made to solr
that are too heavy to have decent turnaround times. As a result the
client may close the connection abruptly, resulting in half closed
connections. You can also try adding search time out to solr queries:
https://issues.apache.org/jira/browse/SOLR-502

On Tue, Jan 10, 2012 at 8:06 AM, Ranveer <ra...@gmail.com> wrote:
> Hi,
>
> I am facing same problem. Did  -Dhttp.maxConnections resolve the problem ?
>
> Please let us know!
>
> regards
> Ranveer
>
>
>
> On Thursday 15 December 2011 11:30 AM, samarth s wrote:
>>
>> Thanks Erick and Mikhail. I'll try this out.
>>
>> On Wed, Dec 14, 2011 at 7:11 PM, Erick Erickson<er...@gmail.com>
>>  wrote:
>>>
>>> I'm guessing (and it's just a guess) that what's happening is that
>>> the container is queueing up your requests while waiting
>>> for the other connections to close, so Mikhail's suggestion
>>> seems like a good idea.
>>>
>>> Best
>>> Erick
>>>
>>> On Wed, Dec 14, 2011 at 12:28 AM, samarth s
>>> <sa...@gmail.com>  wrote:
>>>>
>>>> The updates to the master are user driven, and are needed to be
>>>> visible quickly. Hence, the high frequency of replication. It may be
>>>> that too many replication requests are being handled at a time, but
>>>> why should that result in half closed connections?
>>>>
>>>> On Wed, Dec 14, 2011 at 2:47 AM, Erick Erickson<er...@gmail.com>
>>>>  wrote:
>>>>>
>>>>> Replicating 40 cores every 20 seconds is just *asking* for trouble.
>>>>> How often do your cores change on the master? How big are
>>>>> they? Is there any chance you just have too many cores replicating
>>>>> at once?
>>>>>
>>>>> Best
>>>>> Erick
>>>>>
>>>>> On Tue, Dec 13, 2011 at 3:52 PM, Mikhail Khludnev
>>>>> <mk...@griddynamics.com>  wrote:
>>>>>>
>>>>>> You can try to reuse your connections (prevent them from closing) by
>>>>>> specifying
>>>>>>  -Dhttp.maxConnections=<http://download.oracle.com/javase/1.4.2/docs/guide/net/properties.html>N
>>>>>> in jvm startup params. At client JVM!. Number should be chosen
>>>>>> considering
>>>>>> the number of connection you'd like to keep alive.
>>>>>>
>>>>>> Let me know if it works for you.
>>>>>>
>>>>>> On Tue, Dec 13, 2011 at 2:57 PM, samarth
>>>>>> s<sa...@gmail.com>wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am using solr replication and am experiencing a lot of connections
>>>>>>> in the state CLOSE_WAIT at the master solr server. These disappear
>>>>>>> after a while, but till then the master solr stops responding.
>>>>>>>
>>>>>>> There are about 130 open connections on the master server with the
>>>>>>> client as the slave m/c and all are in the state CLOSE_WAIT. Also,
>>>>>>> the
>>>>>>> client port specified on the master solr server netstat results is
>>>>>>> not
>>>>>>> visible in the netstat results on the client (slave solr) m/c.
>>>>>>>
>>>>>>> Following is my environment:
>>>>>>> - 40 cores in the master solr on m/c 1
>>>>>>> - 40 cores in the slave solr on m/c 2
>>>>>>> - The replication poll interval is 20 seconds.
>>>>>>> - Replication part in solrconfig.xml in the slave solr:
>>>>>>> <requestHandler name="/replication" class="solr.ReplicationHandler">
>>>>>>>           <lst name="slave">
>>>>>>>
>>>>>>>                   <!--fully qualified url for the replication handler
>>>>>>> of master-->
>>>>>>>                   <str
>>>>>>> name="masterUrl">$mastercorename/replication</str>
>>>>>>>
>>>>>>>                   <!--Interval in which the slave should poll master
>>>>>>> .Format is HH:mm:ss . If this is absent slave does not poll
>>>>>>> automatically.
>>>>>>>                                But a fetchindex can be triggered from
>>>>>>> the admin or the http API-->
>>>>>>>                   <str name="pollInterval">00:00:20</str>
>>>>>>>                   <!-- The following values are used when the slave
>>>>>>> connects to the master to download the index files.
>>>>>>>                               Default values implicitly set as 5000ms
>>>>>>> and 10000ms respectively. The user DOES NOT need to specify
>>>>>>>                               these unless the bandwidth is extremely
>>>>>>> low or if there is an extremely high latency-->
>>>>>>>                   <str name="httpConnTimeout">5000</str>
>>>>>>>                   <str name="httpReadTimeout">10000</str>
>>>>>>>          </lst>
>>>>>>>   </requestHandler>
>>>>>>>
>>>>>>> Thanks for any pointers.
>>>>>>>
>>>>>>> --
>>>>>>> Regards,
>>>>>>> Samarth
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sincerely yours
>>>>>> Mikhail Khludnev
>>>>>> Developer
>>>>>> Grid Dynamics
>>>>>> tel. 1-415-738-8644
>>>>>> Skype: mkhludnev
>>>>>> <http://www.griddynamics.com>
>>>>>>  <mk...@griddynamics.com>
>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Samarth
>>
>>
>>
>



-- 
Regards,
Samarth