You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Ben Maurer (JIRA)" <ji...@apache.org> on 2009/02/28 21:48:12 UTC
[jira] Created: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Apply HADOOP-5369 to HBase MapFile
----------------------------------
Key: HBASE-1229
URL: https://issues.apache.org/jira/browse/HBASE-1229
Project: Hadoop HBase
Issue Type: Improvement
Reporter: Ben Maurer
The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Posted by "Ben Maurer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ben Maurer updated HBASE-1229:
------------------------------
Fix Version/s: 0.19.1
> Apply HADOOP-5369 to HBase MapFile
> ----------------------------------
>
> Key: HBASE-1229
> URL: https://issues.apache.org/jira/browse/HBASE-1229
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Ben Maurer
> Fix For: 0.19.1
>
>
> The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack reopened HBASE-1229:
--------------------------
Reopening. Reverting commit. It breaks most unit tests.
> Apply HADOOP-5369 to HBase MapFile
> ----------------------------------
>
> Key: HBASE-1229
> URL: https://issues.apache.org/jira/browse/HBASE-1229
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Ben Maurer
> Assignee: Ben Maurer
> Fix For: 0.19.1
>
> Attachments: 1229.patch
>
>
> The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1229:
-------------------------
Attachment: 1229.patch
Here's a patch that applies to hbase. Tests pass.
Ben, do you want to address the comments raised over in hadoop-5639 before I commit?
> Apply HADOOP-5369 to HBase MapFile
> ----------------------------------
>
> Key: HBASE-1229
> URL: https://issues.apache.org/jira/browse/HBASE-1229
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Ben Maurer
> Fix For: 0.19.1
>
> Attachments: 1229.patch
>
>
> The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-1229.
--------------------------
Resolution: Fixed
Added in latest changes -> "smaller_mapfile.patch -- 2009-03-06 04:50 PM". Resolving.
> Apply HADOOP-5369 to HBase MapFile
> ----------------------------------
>
> Key: HBASE-1229
> URL: https://issues.apache.org/jira/browse/HBASE-1229
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Ben Maurer
> Assignee: Ben Maurer
> Fix For: 0.19.1
>
> Attachments: 1229.patch
>
>
> The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1229:
-------------------------
Fix Version/s: (was: 0.19.1)
0.19.2
Moving to 0.19.2
> Apply HADOOP-5369 to HBase MapFile
> ----------------------------------
>
> Key: HBASE-1229
> URL: https://issues.apache.org/jira/browse/HBASE-1229
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Ben Maurer
> Assignee: Ben Maurer
> Fix For: 0.19.2
>
> Attachments: 1229.patch
>
>
> The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack reassigned HBASE-1229:
----------------------------
Assignee: Ben Maurer
> Apply HADOOP-5369 to HBase MapFile
> ----------------------------------
>
> Key: HBASE-1229
> URL: https://issues.apache.org/jira/browse/HBASE-1229
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Ben Maurer
> Assignee: Ben Maurer
> Fix For: 0.19.1
>
> Attachments: 1229.patch
>
>
> The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-1229.
--------------------------
Resolution: Fixed
Committed Ben's new patch up in the hadoop issue.
> Apply HADOOP-5369 to HBase MapFile
> ----------------------------------
>
> Key: HBASE-1229
> URL: https://issues.apache.org/jira/browse/HBASE-1229
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Ben Maurer
> Assignee: Ben Maurer
> Fix For: 0.19.1
>
> Attachments: 1229.patch
>
>
> The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1229:
-------------------------
Fix Version/s: (was: 0.19.2)
0.19.1
Restoring to 0.19.1 -- moved wrong issue by mistake
> Apply HADOOP-5369 to HBase MapFile
> ----------------------------------
>
> Key: HBASE-1229
> URL: https://issues.apache.org/jira/browse/HBASE-1229
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Ben Maurer
> Assignee: Ben Maurer
> Fix For: 0.19.1
>
> Attachments: 1229.patch
>
>
> The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12679720#action_12679720 ]
stack commented on HBASE-1229:
------------------------------
Here's the issue:
2009-03-06 12:45:19,289 ERROR [main] master.HMaster(267): bootstrap
org.apache.hadoop.hbase.DroppedSnapshotException: region: -ROOT-,,0
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:896)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:433)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:364)
at org.apache.hadoop.hbase.master.HMaster.bootstrap(HMaster.java:261)
at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:216)
at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
at org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:57)
at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:51)
at org.apache.hadoop.hbase.HBaseClusterTestCase.hBaseClusterSetup(HBaseClusterTestCase.java:88)
at org.apache.hadoop.hbase.HBaseClusterTestCase.setUp(HBaseClusterTestCase.java:125)
at junit.framework.TestCase.runBare(TestCase.java:125)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:421)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:912)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:766)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
at org.apache.hadoop.hbase.io.MapFile$Reader.midKey(MapFile.java:399)
at org.apache.hadoop.hbase.io.HBaseMapFile$HBaseReader.<init>(HBaseMapFile.java:87)
at org.apache.hadoop.hbase.io.BloomFilterMapFile$Reader.<init>(BloomFilterMapFile.java:68)
at org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:443)
at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:720)
at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:694)
at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:630)
at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:881)
... 20 more
> Apply HADOOP-5369 to HBase MapFile
> ----------------------------------
>
> Key: HBASE-1229
> URL: https://issues.apache.org/jira/browse/HBASE-1229
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Ben Maurer
> Assignee: Ben Maurer
> Fix For: 0.19.1
>
> Attachments: 1229.patch
>
>
> The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-1229.
--------------------------
Resolution: Fixed
Resolved. I applied the last patch pasted by Ben up in the HADOOP issue. Did not apply TRUNK. We don't use mapfile any more.
> Apply HADOOP-5369 to HBase MapFile
> ----------------------------------
>
> Key: HBASE-1229
> URL: https://issues.apache.org/jira/browse/HBASE-1229
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Ben Maurer
> Fix For: 0.19.1
>
> Attachments: 1229.patch
>
>
> The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HBASE-1229) Apply HADOOP-5369 to HBase MapFile
Posted by "Ben Maurer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ben Maurer reopened HBASE-1229:
-------------------------------
I made another change to the hadoop patch... i think this is the last time :-).
> Apply HADOOP-5369 to HBase MapFile
> ----------------------------------
>
> Key: HBASE-1229
> URL: https://issues.apache.org/jira/browse/HBASE-1229
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Ben Maurer
> Assignee: Ben Maurer
> Fix For: 0.19.1
>
> Attachments: 1229.patch
>
>
> The patch in HADOOP-5369 reduces the memory overhead of mapfile indexes -- especially when block compression is used. It should be applied to the hbase copy of MapFile.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.