[jira] [Reopened] (HBASE-21551) Memory leak when use scan with STREAM at server side

["Sean Busbey (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-21551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Busbey reopened HBASE-21551:
---------------------------------

reopening so I can add a release note

> Memory leak when use scan with STREAM at server side
> ----------------------------------------------------
>
>                 Key: HBASE-21551
>                 URL: https://issues.apache.org/jira/browse/HBASE-21551
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>            Reporter: Zheng Hu
>            Assignee: Zheng Hu
>            Priority: Blocker
>             Fix For: 3.0.0, 2.2.0, 2.1.2, 2.0.4
>
>         Attachments: HBASE-21551.v1.patch, HBASE-21551.v2.patch, HBASE-21551.v3.patch, heap-dump.jpg
>
>
> We open the RegionServerScanner with STREAM as following: 
> {code}
> RegionScannerImpl#initializeScanners
>       |---> HStore#getScanner
>                     |----------> StoreScanner()
>                                         |-------> StoreFileScanner#getScannersForStoreFiles
>                                                           |------> HStoreFile#getStreamScanner      #1
> {code}
> In #1,  we put the StoreFileReader into  a concurrent hash map streamReaders, but not remove the StreamReader from streamReaders until closing the store file. 
> So if we  scan with stream with  so many times, the streamReaders hash map will be exploded.   we can see the heap dump in the attached heap-dump.jpg. 
> I found this bug, because when i benchmark the scan performance by using YCSB in a cluster (heap size of RS is 50g),  the Rs was easy to occur a long time full gc ( ~ 110 sec)....



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)