[jira] [Commented] (SOLR-4509) Disable Stale Check - Distributed Search (Performance)

["Mark Miller (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221753#comment-14221753 ] 

Mark Miller commented on SOLR-4509:
-----------------------------------

My biggest hesitation on this patch is:

{code}
+    // NOTE: The sweeper task is assuming hard-coded Jetty max-idle of 50s.
+    final Runnable sweeper = new Runnable() {
+            public void run() {
+                mgr.closeIdleConnections(40, TimeUnit.SECONDS);
+            }
+        };
+    final ScheduledExecutorService stp = Executors.newScheduledThreadPool(1);
+    stp.scheduleWithFixedDelay(sweeper, 5, 5, TimeUnit.SECONDS);
{code}

> Disable Stale Check - Distributed Search (Performance)
> ------------------------------------------------------
>
>                 Key: SOLR-4509
>                 URL: https://issues.apache.org/jira/browse/SOLR-4509
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>         Environment: 5 node SmartOS cluster (all nodes living in same global zone - i.e. same physical machine)
>            Reporter: Ryan Zezeski
>            Priority: Minor
>         Attachments: IsStaleTime.java, SOLR-4509-4_4_0.patch, SOLR-4509.patch, baremetal-stale-nostale-med-latency.dat, baremetal-stale-nostale-med-latency.svg, baremetal-stale-nostale-throughput.dat, baremetal-stale-nostale-throughput.svg
>
>
> By disabling the Apache HTTP Client stale check I've witnessed a 2-4x increase in throughput and reduction of over 100ms.  This patch was made in the context of a project I'm leading, called Yokozuna, which relies on distributed search.
> Here's the patch on Yokozuna: https://github.com/rzezeski/yokozuna/pull/26
> Here's a write-up I did on my findings: http://www.zinascii.com/2013/solr-distributed-search-and-the-stale-check.html
> I'm happy to answer any questions or make changes to the patch to make it acceptable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org