[jira] [Updated] (SOLR-16129) Solr specific InputStreamResponseListener to prevent client threads from hanging forever

["Chris M. Hostetter (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SOLR-16129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris M. Hostetter updated SOLR-16129:
--------------------------------------
    Attachment: SOLR-16129.patch
        Status: Open  (was: Open)

The attached patch implements this change pretty closely to what i suggested in the linked jira...
 * new forked {{InputStreamResponseListener}} impl
 ** subclassing wasn't an option due to private vars
 * mandatory {{maxWaitLimit}} constructor arg
 ** {{Http2SolrClient}} passes it's idelTimeout for this arg
 * optional {{setRequestTimeout}}

 ** defaults to 1 hour
 ** Http2SolrClient passes {{30ms + (2 * timeAllowed)}} if timeAllowed is used
 *** this gives the server enough time to still respond with partial results if timeAllowed is exceeded, but prevents the client thread from waiting forever
 * when {{InputStream}} reads block due to incomplete response data from remote client:
 ** call fails with {{IOException}} if {{requestTimeout}} exceeded
 ** wait loop uses max of {{maxWaitLimit}} vs (remaining) {{requestTimeout}} in individual wait calls

All existing tests pass with this change ... I plan to add a new unit test of the Listener that shows {{{}getInputStream().read(){}}}:
 * won't hang forever if the low level client code never calls {{onSuccess}} or {{onFailure}}
 * won't wait beyond {{requestTimeout}} if an infinite stream of bytes is being passed into {{onContent}}

----
[~noblepaul] / [~ichattopadhyaya] - since you guys seemed to be able to reliably reproduce the problem described in SOLR-16099, it would be great if you could try out this patch and confirm that it prevents the client threads from hanging forever.

> Solr specific InputStreamResponseListener to prevent client threads from hanging forever
> ----------------------------------------------------------------------------------------
>
>                 Key: SOLR-16129
>                 URL: https://issues.apache.org/jira/browse/SOLR-16129
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Chris M. Hostetter
>            Assignee: Chris M. Hostetter
>            Priority: Major
>         Attachments: SOLR-16129.patch
>
>
> This issue tracks the implementation of workaround I suggested for SOLR-16099 - it does not _fix_ the underlying bug (which as of this writting doesn't have an identified root cause) but it does ensure that client threads which encounter the bug won't hang forever...
> {quote}One thing we may want to consider (in Solr) is replacing our usage of {{InputStreamResponseListener}} with a variant implementation that uses a "timeout" instead of an unlimited {{wait()}} (along the lines of a [spin-off jetty enhancement issue|https://github.com/eclipse/jetty.project/issues/7259] one of the jetty devs filed). We could probably (with some effort) tweak the impacted Solr APIs to propogate the (remaining) {{timeAllowed}} (if that option was specified) down to this class – and/or have an "extreme" default (ie: 30min) just to prevent threads from sticking around forever.
> {quote}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org