You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org> on 2010/10/27 01:52:19 UTC

[jira] Created: (HBASE-3158) Bloom File Writes Broken if keySize is large

Bloom File Writes Broken if keySize is large
--------------------------------------------

                 Key: HBASE-3158
                 URL: https://issues.apache.org/jira/browse/HBASE-3158
             Project: HBase
          Issue Type: Bug
    Affects Versions: 0.90.0
            Reporter: Nicolas Spiegelberg
            Assignee: Nicolas Spiegelberg
            Priority: Blocker


Yesterday, on our cluster, a region compact() kept crashing at giving this stack trace

2010-10-25 08:48:28,330 ERROR org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction failed for region MailBox_dark_launch_2010_10-04,4b64b600,1286302852538.9183a3b91ebd289bab7724d028cffa69.
java.lang.IllegalArgumentException: maxValue must be > 0
at org.apache.hadoop.hbase.util.ByteBloomFilter.sanityCheck(ByteBloomFilter.java:170)
at org.apache.hadoop.hbase.util.ByteBloomFilter.<init>(ByteBloomFilter.java:156)
at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.<init>(StoreFile.java:707)
at org.apache.hadoop.hbase.regionserver.StoreFile.createWriter(StoreFile.java:566)
at org.apache.hadoop.hbase.regionserver.Store.createWriterInTmp(Store.java:504)
at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:817)
at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:678)
at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:842)
at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:793)
at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:103)

The problem is that we are oveflowing "int" for "bitSize". The number of keys is about 272M, and we are using about 11 bit per key. So, bitSize ends up being > 2G..

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-3158) Bloom File Writes Broken if keySize is large

Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925257#action_12925257 ] 

Nicolas Spiegelberg commented on HBASE-3158:
--------------------------------------------

lol.  I was trying to debug an assert failure that I had during unit testing.  You can obviously take that conditional out :)

> Bloom File Writes Broken if keySize is large
> --------------------------------------------
>
>                 Key: HBASE-3158
>                 URL: https://issues.apache.org/jira/browse/HBASE-3158
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Blocker
>             Fix For: 0.90.0
>
>         Attachments: HBASE-3158.patch
>
>
> Yesterday, on our cluster, a region compact() kept crashing at giving this stack trace
> 2010-10-25 08:48:28,330 ERROR org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction failed for region MailBox_dark_launch_2010_10-04,4b64b600,1286302852538.9183a3b91ebd289bab7724d028cffa69.
> java.lang.IllegalArgumentException: maxValue must be > 0
> at org.apache.hadoop.hbase.util.ByteBloomFilter.sanityCheck(ByteBloomFilter.java:170)
> at org.apache.hadoop.hbase.util.ByteBloomFilter.<init>(ByteBloomFilter.java:156)
> at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.<init>(StoreFile.java:707)
> at org.apache.hadoop.hbase.regionserver.StoreFile.createWriter(StoreFile.java:566)
> at org.apache.hadoop.hbase.regionserver.Store.createWriterInTmp(Store.java:504)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:817)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:678)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:842)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:793)
> at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:103)
> The problem is that we are oveflowing "int" for "bitSize". The number of keys is about 272M, and we are using about 11 bit per key. So, bitSize ends up being > 2G..

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-3158) Bloom File Writes Broken if keySize is large

Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nicolas Spiegelberg updated HBASE-3158:
---------------------------------------

    Status: Patch Available  (was: Open)

> Bloom File Writes Broken if keySize is large
> --------------------------------------------
>
>                 Key: HBASE-3158
>                 URL: https://issues.apache.org/jira/browse/HBASE-3158
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Blocker
>         Attachments: HBASE-3158.patch
>
>
> Yesterday, on our cluster, a region compact() kept crashing at giving this stack trace
> 2010-10-25 08:48:28,330 ERROR org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction failed for region MailBox_dark_launch_2010_10-04,4b64b600,1286302852538.9183a3b91ebd289bab7724d028cffa69.
> java.lang.IllegalArgumentException: maxValue must be > 0
> at org.apache.hadoop.hbase.util.ByteBloomFilter.sanityCheck(ByteBloomFilter.java:170)
> at org.apache.hadoop.hbase.util.ByteBloomFilter.<init>(ByteBloomFilter.java:156)
> at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.<init>(StoreFile.java:707)
> at org.apache.hadoop.hbase.regionserver.StoreFile.createWriter(StoreFile.java:566)
> at org.apache.hadoop.hbase.regionserver.Store.createWriterInTmp(Store.java:504)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:817)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:678)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:842)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:793)
> at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:103)
> The problem is that we are oveflowing "int" for "bitSize". The number of keys is about 272M, and we are using about 11 bit per key. So, bitSize ends up being > 2G..

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-3158) Bloom File Writes Broken if keySize is large

Posted by "stack (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack updated HBASE-3158:
-------------------------

       Resolution: Fixed
    Fix Version/s: 0.90.0
     Hadoop Flags: [Reviewed]
           Status: Resolved  (was: Patch Available)

Thanks for the fix Nicolas (I like the BBQ exception).

> Bloom File Writes Broken if keySize is large
> --------------------------------------------
>
>                 Key: HBASE-3158
>                 URL: https://issues.apache.org/jira/browse/HBASE-3158
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Blocker
>             Fix For: 0.90.0
>
>         Attachments: HBASE-3158.patch
>
>
> Yesterday, on our cluster, a region compact() kept crashing at giving this stack trace
> 2010-10-25 08:48:28,330 ERROR org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction failed for region MailBox_dark_launch_2010_10-04,4b64b600,1286302852538.9183a3b91ebd289bab7724d028cffa69.
> java.lang.IllegalArgumentException: maxValue must be > 0
> at org.apache.hadoop.hbase.util.ByteBloomFilter.sanityCheck(ByteBloomFilter.java:170)
> at org.apache.hadoop.hbase.util.ByteBloomFilter.<init>(ByteBloomFilter.java:156)
> at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.<init>(StoreFile.java:707)
> at org.apache.hadoop.hbase.regionserver.StoreFile.createWriter(StoreFile.java:566)
> at org.apache.hadoop.hbase.regionserver.Store.createWriterInTmp(Store.java:504)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:817)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:678)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:842)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:793)
> at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:103)
> The problem is that we are oveflowing "int" for "bitSize". The number of keys is about 272M, and we are using about 11 bit per key. So, bitSize ends up being > 2G..

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-3158) Bloom File Writes Broken if keySize is large

Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nicolas Spiegelberg updated HBASE-3158:
---------------------------------------

    Attachment: HBASE-3158.patch

Also, added "io.storefile.bloom.max.keys" to control the max keys allowed in any bloom

> Bloom File Writes Broken if keySize is large
> --------------------------------------------
>
>                 Key: HBASE-3158
>                 URL: https://issues.apache.org/jira/browse/HBASE-3158
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Blocker
>         Attachments: HBASE-3158.patch
>
>
> Yesterday, on our cluster, a region compact() kept crashing at giving this stack trace
> 2010-10-25 08:48:28,330 ERROR org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction failed for region MailBox_dark_launch_2010_10-04,4b64b600,1286302852538.9183a3b91ebd289bab7724d028cffa69.
> java.lang.IllegalArgumentException: maxValue must be > 0
> at org.apache.hadoop.hbase.util.ByteBloomFilter.sanityCheck(ByteBloomFilter.java:170)
> at org.apache.hadoop.hbase.util.ByteBloomFilter.<init>(ByteBloomFilter.java:156)
> at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.<init>(StoreFile.java:707)
> at org.apache.hadoop.hbase.regionserver.StoreFile.createWriter(StoreFile.java:566)
> at org.apache.hadoop.hbase.regionserver.Store.createWriterInTmp(Store.java:504)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:817)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:678)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:842)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:793)
> at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:103)
> The problem is that we are oveflowing "int" for "bitSize". The number of keys is about 272M, and we are using about 11 bit per key. So, bitSize ends up being > 2G..

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-3158) Bloom File Writes Broken if keySize is large

Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925204#action_12925204 ] 

Nicolas Spiegelberg commented on HBASE-3158:
--------------------------------------------

Two fixes to make:

1) Catch IllegalArgumentExceptions thrown by the BloomFilter and just continue writing the StoreFile without blooms instead of crashing
2) ByteBloomFilter.<init>.bitSize should be a long and we should be able to handle large bloom sizes
3) We cannot handle byteSize > MAX_INT because java doesn't allow you to allocate arrays that large.  Throw an IAE in that case

> Bloom File Writes Broken if keySize is large
> --------------------------------------------
>
>                 Key: HBASE-3158
>                 URL: https://issues.apache.org/jira/browse/HBASE-3158
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Blocker
>
> Yesterday, on our cluster, a region compact() kept crashing at giving this stack trace
> 2010-10-25 08:48:28,330 ERROR org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction failed for region MailBox_dark_launch_2010_10-04,4b64b600,1286302852538.9183a3b91ebd289bab7724d028cffa69.
> java.lang.IllegalArgumentException: maxValue must be > 0
> at org.apache.hadoop.hbase.util.ByteBloomFilter.sanityCheck(ByteBloomFilter.java:170)
> at org.apache.hadoop.hbase.util.ByteBloomFilter.<init>(ByteBloomFilter.java:156)
> at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.<init>(StoreFile.java:707)
> at org.apache.hadoop.hbase.regionserver.StoreFile.createWriter(StoreFile.java:566)
> at org.apache.hadoop.hbase.regionserver.Store.createWriterInTmp(Store.java:504)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:817)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:678)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:842)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:793)
> at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:103)
> The problem is that we are oveflowing "int" for "bitSize". The number of keys is about 272M, and we are using about 11 bit per key. So, bitSize ends up being > 2G..

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.