You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org> on 2016/07/12 18:59:20 UTC
[jira] [Updated] (SOLR-9290) TCP-connections in CLOSE_WAIT spikes
during heavy indexing when SSL is enabled
[ https://issues.apache.org/jira/browse/SOLR-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-9290:
----------------------------------------
Attachment: SOLR-9290-debug.patch
setup-solr.sh
The setup-solr.sh is small script that I use to setup different versions with SSL enabled and debug logging enabled for httpclient.
The debug logging looks like the following:
{code}
2016-07-12 13:25:05.692 DEBUG (httpShardExecutor-4-thread-7-processing-x:xyz_shard3_replica1 r:core_node5 https:////127.0.1.1:8983//solr//xyz_shard3_replica2// n:127.0.1.1:8984_solr s:shard3 c:xyz [https:////127.0.1.1:8983//solr//xyz_shard3_replica2//]) [c:xyz s:shard3 r:core_node5 x:xyz_shard3_replica1] o.a.h.i.c.PoolingClientConnectionManager Connection leased: [id: 17][route: {s}->https://127.0.1.1:8983][total kept alive: 0; route allocated: 5 of 100000; total allocated: 5 of 100000]
...
2016-07-12 13:25:05.791 DEBUG (recoveryExecutor-3-thread-4-processing-n:127.0.1.1:8984_solr x:xyz_shard1_replica1 s:shard1 c:xyz r:core_node2) [c:xyz s:shard1 r:core_node2 x:xyz_shard1_replica1] o.a.h.i.c.PoolingClientConnectionManager Connection released: [id: 17][route: {s}->https://127.0.1.1:8983][total kept alive: 8; route allocated: 8 of 100000; total allocated: 8 of 100000]
{code}
The attached SOLR-9290-debug.patch applies to 5.3.x and changes HttpSolrClient to log the connection details including the client port number for each request.
> TCP-connections in CLOSE_WAIT spikes during heavy indexing when SSL is enabled
> ------------------------------------------------------------------------------
>
> Key: SOLR-9290
> URL: https://issues.apache.org/jira/browse/SOLR-9290
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Affects Versions: 5.5.1, 5.5.2
> Reporter: Anshum Gupta
> Priority: Critical
> Attachments: SOLR-9290-debug.patch, setup-solr.sh
>
>
> Heavy indexing on Solr with SSL leads to a lot of connections in CLOSE_WAIT state.
> At my workplace, we have seen this issue only with 5.5.1 and could not reproduce it with 5.4.1 but from my conversation with Shalin, he knows of users with 5.3.1 running into this issue too.
> Here's an excerpt from the email [~shaie] sent to the mailing list (about what we see:
> {quote}
> 1) It consistently reproduces on 5.5.1, but *does not* reproduce on 5.4.1
> 2) It does not reproduce when SSL is disabled
> 3) Restarting the Solr process (sometimes both need to be restarted), the
> count drops to 0, but if indexing continues, they climb up again
> When it does happen, Solr seems stuck. The leader cannot talk to the
> replica, or vice versa, the replica is usually put in DOWN state and
> there's no way to fix it besides restarting the JVM.
> {quote}
> Here's the mail thread: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201607.mbox/%3C46cc66220a8143dc903fa34e792059c4@vp-exc01.dips.local%3E
> Creating this issue so we could track this and have more people comment on what they see.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org