[jira] [Commented] (LUCENE-4995) Remove the strong reference of CompressingStoredFieldsReader on the decompression buffer

["Aditya (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-4995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13677265#comment-13677265 ] 

Aditya commented on LUCENE-4995:
--------------------------------

We are currently facing this issue when we upgraded Solr from 3.5 to 4.x. Our documents have large multiValued fields and we do see huge Heap which is not claimed back after GC. 3.5 with same set of documents works very well and the heap usage is around 50 to 65% (on long run with no replication)

Some specs we used about test. (no replication happening anytime during our test)
#CPU 6 Cores
RAM 16Gb
Heap 8GB (tried with 10 and 12 GB too)
Index size 4.6GB
Solr version 4.2.1 (on Stand Alone M/c. Our Production implementation is Master/Slave)
GC Policy used is CMS (tried G1GC too)


On analyzing the heap we see that the largest object in the heap is the byteRef tracing it back it points to CompressedStoredFieldReader. We are looking into the patch above and test if that helps

if you need more information. I can provide. 

thanks
Aditya  
                
> Remove the strong reference of CompressingStoredFieldsReader on the decompression buffer
> ----------------------------------------------------------------------------------------
>
>                 Key: LUCENE-4995
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4995
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Adrien Grand
>            Assignee: Adrien Grand
>         Attachments: LUCENE-4995.patch
>
>
> CompressingStoredFieldsReader has a strong reference on the buffer it uses for decompression. Although it makes the reader able to reuse this buffer, this can trigger high memory usage in case some documents are very large. Creating this buffer on demand would help give memory back to the JVM.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org