You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2011/05/04 05:50:03 UTC
[jira] [Created] (HBASE-3852) ThriftServer leaks scanners
ThriftServer leaks scanners
---------------------------
Key: HBASE-3852
URL: https://issues.apache.org/jira/browse/HBASE-3852
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.2
Reporter: Jean-Daniel Cryans
Priority: Critical
Fix For: 0.92.0
The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205805#comment-13205805 ]
Zhihong Yu commented on HBASE-3852:
-----------------------------------
{code}
+ timestamp = System.currentTimeMillis();
{code}
We have EnvironmentEdgeManager which provides the above service.
{code}
+ for (Integer key: toremove)
+ removeScanner(key);
{code}
Please use curly braces around the removeScanner() call.
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.94.0
>
> Attachments: 3852.txt, thrift2-scanner.patch
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404542#comment-13404542 ]
Lars Hofhansl commented on HBASE-3852:
--------------------------------------
Any update on this?
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Components: thrift
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.94.1
>
> Attachments: 3852.txt, thrift2-scanner.patch
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277955#comment-13277955 ]
Lars Hofhansl commented on HBASE-3852:
--------------------------------------
In the remove case, should we also close the scanner?
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Components: thrift
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.94.1
>
> Attachments: 3852.txt, thrift2-scanner.patch
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-3852) ThriftServer leaks scanners
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu reassigned HBASE-3852:
-----------------------------
Assignee: Ted Yu
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.92.0
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-3852:
---------------------------------
Fix Version/s: (was: 0.94.0)
0.94.1
Let's try for 0.94.1
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.94.1
>
> Attachments: 3852.txt, thrift2-scanner.patch
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-3852:
--------------------------
Fix Version/s: (was: 0.92.0)
0.94.0
Moving out of 0.92 for now.
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.94.0
>
> Attachments: 3852.txt
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030910#comment-13030910 ]
Alex Newman commented on HBASE-3852:
------------------------------------
Should we close scanners on disconnect from the thrift server?
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.92.0
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054616#comment-13054616 ]
Jean-Daniel Cryans commented on HBASE-3852:
-------------------------------------------
bq. do you happen to know how many scanners were closed by ScannerCleaner in the past month ?
I used to print that out but it was really spammy. Probably tens of thousands.
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3852.txt
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030920#comment-13030920 ]
Jean-Daniel Cryans commented on HBASE-3852:
-------------------------------------------
@Alex, I'm not sure I understand what you're saying, by "on disconnect" you mean when client is done with the server and closes the socket? How do you know that? And what if the client was scanning but then never comes back?
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.92.0
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054565#comment-13054565 ]
Ted Yu commented on HBASE-3852:
-------------------------------
I was thinking about timeout-driven expiration policy.
SU's implementation looks nice. We can introduce a new parameter (sigh) to control whether ScannerCleaner is instantiated at startup.
@J-D: do you happen to know how many scanners were closed by ScannerCleaner in the past month ?
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3852.txt
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030922#comment-13030922 ]
Alex Newman commented on HBASE-3852:
------------------------------------
Yes, when the client closes the connection to the thrift server. I'm just trying to understand when you think the socket should be cleaned up.
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.92.0
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054006#comment-13054006 ]
Ted Yu commented on HBASE-3852:
-------------------------------
In scannerGetList():
{code}
if (null == results) {
return new ArrayList<TRowResult>();
}
{code}
Can we close the scanner before returning ?
We still leave the scanner in scannerMap.
HTable.ClientScanner would check whether the scanner has been closed.
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.92.0
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
Posted by "Jonathan Hsieh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh updated HBASE-3852:
----------------------------------
Component/s: thrift
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Components: thrift
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.94.1
>
> Attachments: 3852.txt, thrift2-scanner.patch
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-3852:
---------------------------------
Fix Version/s: (was: 0.94.1)
0.94.2
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Components: thrift
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.94.2
>
> Attachments: 3852.txt, thrift2-scanner.patch
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
Posted by "Bob Copeland (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bob Copeland updated HBASE-3852:
--------------------------------
Attachment: thrift2-scanner.patch
We hit this running thrift2 in production. With a lot of scanners buffering a lot (100) of rows per call, thrift servers would OOME in no time. This patch just periodically expires old ones. I can respin/rework the patch if the approach is sound.
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.94.0
>
> Attachments: 3852.txt, thrift2-scanner.patch
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-3852:
--------------------------
Attachment: 3852.txt
TestThriftServer passes.
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3852.txt
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205827#comment-13205827 ]
stack commented on HBASE-3852:
------------------------------
Patch looks good. Thanks Bob. Suggest picking up the coding convention of surrounding class. Rather than hardcode 60 seconds, maybe use value of hbase.regionserver.lease.period gotten from Configuration instance?
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.94.0
>
> Attachments: 3852.txt, thrift2-scanner.patch
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234506#comment-13234506 ]
Lars Hofhansl commented on HBASE-3852:
--------------------------------------
Can we commit this? Or should this jira move to 0.96?
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.94.0
>
> Attachments: 3852.txt, thrift2-scanner.patch
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-3852:
---------------------------------
Fix Version/s: (was: 0.94.3)
0.94.4
And on to 0.94.4
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Components: Thrift
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Fix For: 0.94.4
>
> Attachments: 3852.txt, thrift2-scanner.patch
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028581#comment-13028581 ]
stack commented on HBASE-3852:
------------------------------
Good one J-D. How'd you figure it? Did you restart the thrift server? Jack would be interested in this one?
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.92.0
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054544#comment-13054544 ]
Jean-Daniel Cryans commented on HBASE-3852:
-------------------------------------------
I'm not sure it fixes the problem, since the issue here is about users not getting back to close even tho there's still more data. We patched in something at SU that we've been running for more than a month, but it is specific to our usecase. Anyways, feel free to inspire yourself with those two commits:
https://github.com/stumbleupon/hbase/commit/7bcc63ee22f7e0218fb7225f387ffc0bd2279f59#src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java
https://github.com/stumbleupon/hbase/commit/3d08cc8ad5abfec11ee0340d5a75a2b308854e17#src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3852.txt
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-3852) ThriftServer leaks scanners
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu reassigned HBASE-3852:
-----------------------------
Assignee: (was: Ted Yu)
SU has a solution already.
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3852.txt
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3852) ThriftServer leaks scanners
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482974#comment-13482974 ]
Lars Hofhansl commented on HBASE-3852:
--------------------------------------
@J-D: Are you still working on this?
> ThriftServer leaks scanners
> ---------------------------
>
> Key: HBASE-3852
> URL: https://issues.apache.org/jira/browse/HBASE-3852
> Project: HBase
> Issue Type: Bug
> Components: Thrift
> Affects Versions: 0.90.2
> Reporter: Jean-Daniel Cryans
> Fix For: 0.94.3
>
> Attachments: 3852.txt, thrift2-scanner.patch
>
>
> The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira