[jira] [Commented] (SOLR-9248) HttpSolrClient not compatible with compression option

["Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-9248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375091#comment-15375091 ] 

Shalin Shekhar Mangar commented on SOLR-9248:
---------------------------------------------

Working on SOLR-9290, I remembered about this issue. I think the problem here is that GzipDecompressingEntity (and DeflateDecompressingEntity) do not adhere to the contract laid out for HttpEntity#getContent which states that:
{code}
     * Returns a content stream of the entity.
     * {@link #isRepeatable Repeatable} entities are expected
     * to create a new instance of {@link InputStream} for each invocation
     * of this method and therefore can be consumed multiple times.
     * Entities that are not {@link #isRepeatable repeatable} are expected
     * to return the same {@link InputStream} instance and therefore
     * may not be consumed more than once.
{code}

So both those classes should always return the same object from the getContent method as long as the wrapped entity is non-repeatable and a new InputStream otherwise.

Also, I'd say it is a good idea in general to avoid calling getContent a second time just to be able to consume it. So even though all entities we actually use in Solr are non-repeatable, Solr code should just make one call to getContent keep the InputStream instance around for the consumption to avoid these kind of bugs.

> HttpSolrClient not compatible with compression option
> -----------------------------------------------------
>
>                 Key: SOLR-9248
>                 URL: https://issues.apache.org/jira/browse/SOLR-9248
>             Project: Solr
>          Issue Type: Bug
>          Components: SolrJ
>    Affects Versions: 5.5, 5.5.1
>            Reporter: Gary Lee
>         Attachments: CompressedConnectionTest.java
>
>
> Since Solr 5.5, using the compression option (solrClient.setAllowCompression(true)) causes the HTTP client to quickly run out of connections in the connection pool. After debugging through this, we found that the GZIPInputStream is incompatible with changes to how the response input stream is closed in 5.5. It is at this point when the GZIPInputStream throws an EOFException, and while this is silently eaten up, the net effect is that the stream is never closed, leaving the connection open. After a number of requests, the pool is exhausted and no further requests can be served.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org