You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2011/05/27 08:34:47 UTC
[jira] [Commented] (HBASE-3421) Very wide rows -- 30M plus -- cause
us OOME
[ https://issues.apache.org/jira/browse/HBASE-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040082#comment-13040082 ]
Todd Lipcon commented on HBASE-3421:
------------------------------------
I think I'm seeing this on one workload as well - I added a "stats" command for the HFile tool, then ran it on a problematic 1.5GB gzip-compressed HFile that won't compact due to OOME.
Key length: count: 272358037 min: 59 max: 2259 mean: 59.620136732003246
Val length: count: 272358037 min: 0 max: 0 mean: 0.0
Row size (bytes): count: 1287745 min: 67 max: 15615127736 mean: 14301.657317248368
Row size (columns): count: 1287745 min: 1 max: 233061608 mean: 211.49997631518661
so in this case there's actually a row with 233 million columns which takes up 15GB uncompressed!
> Very wide rows -- 30M plus -- cause us OOME
> -------------------------------------------
>
> Key: HBASE-3421
> URL: https://issues.apache.org/jira/browse/HBASE-3421
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.0
> Reporter: stack
>
> From the list, see 'jvm oom' in http://mail-archives.apache.org/mod_mbox/hbase-user/201101.mbox/browser, it looks like wide rows -- 30M or so -- causes OOME during compaction. We should check it out. Can the scanner used during compactions use the 'limit' when nexting? If so, this should save our OOME'ing (or, we need to add to the next a max size rather than count of KVs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira