You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Clint Morgan (JIRA)" <ji...@apache.org> on 2009/09/22 19:52:16 UTC
[jira] Created: (HBASE-1858) Master can't split logs created by
THBase
Master can't split logs created by THBase
-----------------------------------------
Key: HBASE-1858
URL: https://issues.apache.org/jira/browse/HBASE-1858
Project: Hadoop HBase
Issue Type: Bug
Components: master
Affects Versions: 0.20.0
Reporter: Clint Morgan
Assignee: Clint Morgan
Fix For: 0.20.1, 0.21.0
When master tries to split logs created by THbase, it fails because it tries to read in the wrong key type. (THBase subclasses HLogKey to add fields to the key).
2009-09-16 09:03:01,943 WARN org.apache.hadoop.hbase.regionserver.HLog:
Exception processing
hdfs://domU-12-31-39-07-CC-A2.compute-1.internal:9000/hbase/.logs/domU-12-31-39-07-CC-A2.compute-1.internal,60020,1253103101743/hlog.dat.1253103102168
-- continuing. Possible DATA LOSS!
java.io.IOException: wrong key class:
org.apache.hadoop.hbase.regionserver.HLogKey is not class
org.apache.hadoop.hbase.regionserver.transactional.THLogKey
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1824)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876)
at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:880)
at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:802)
at
org.apache.hadoop.hbase.master.ProcessServerShutdown.process(ProcessServerShutdown.java:274)
at
org.apache.hadoop.hbase.master.HMaster.processToDoQueue(HMaster.java:492)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:426)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1858) Master can't split logs created by
THBase
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-1858.
--------------------------
Resolution: Fixed
Applied to branch. Closing. Thanks for the patch Clint.
> Master can't split logs created by THBase
> -----------------------------------------
>
> Key: HBASE-1858
> URL: https://issues.apache.org/jira/browse/HBASE-1858
> Project: Hadoop HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.20.0
> Reporter: Clint Morgan
> Assignee: Clint Morgan
> Fix For: 0.20.1, 0.21.0
>
> Attachments: 1858-v2.patch, 1858.patch
>
>
> When master tries to split logs created by THbase, it fails because it tries to read in the wrong key type. (THBase subclasses HLogKey to add fields to the key).
> 2009-09-16 09:03:01,943 WARN org.apache.hadoop.hbase.regionserver.HLog:
> Exception processing
> hdfs://domU-12-31-39-07-CC-A2.compute-1.internal:9000/hbase/.logs/domU-12-31-39-07-CC-A2.compute-1.internal,60020,1253103101743/hlog.dat.1253103102168
> -- continuing. Possible DATA LOSS!
> java.io.IOException: wrong key class:
> org.apache.hadoop.hbase.regionserver.HLogKey is not class
> org.apache.hadoop.hbase.regionserver.transactional.THLogKey
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1824)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:880)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:802)
> at
> org.apache.hadoop.hbase.master.ProcessServerShutdown.process(ProcessServerShutdown.java:274)
> at
> org.apache.hadoop.hbase.master.HMaster.processToDoQueue(HMaster.java:492)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:426)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1858) Master can't split logs created by
THBase
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758491#action_12758491 ]
stack commented on HBASE-1858:
------------------------------
Do you have a fix for this Clint?
> Master can't split logs created by THBase
> -----------------------------------------
>
> Key: HBASE-1858
> URL: https://issues.apache.org/jira/browse/HBASE-1858
> Project: Hadoop HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.20.0
> Reporter: Clint Morgan
> Assignee: Clint Morgan
> Fix For: 0.20.1, 0.21.0
>
>
> When master tries to split logs created by THbase, it fails because it tries to read in the wrong key type. (THBase subclasses HLogKey to add fields to the key).
> 2009-09-16 09:03:01,943 WARN org.apache.hadoop.hbase.regionserver.HLog:
> Exception processing
> hdfs://domU-12-31-39-07-CC-A2.compute-1.internal:9000/hbase/.logs/domU-12-31-39-07-CC-A2.compute-1.internal,60020,1253103101743/hlog.dat.1253103102168
> -- continuing. Possible DATA LOSS!
> java.io.IOException: wrong key class:
> org.apache.hadoop.hbase.regionserver.HLogKey is not class
> org.apache.hadoop.hbase.regionserver.transactional.THLogKey
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1824)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:880)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:802)
> at
> org.apache.hadoop.hbase.master.ProcessServerShutdown.process(ProcessServerShutdown.java:274)
> at
> org.apache.hadoop.hbase.master.HMaster.processToDoQueue(HMaster.java:492)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:426)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1858) Master can't split logs created by
THBase
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759945#action_12759945 ]
stack commented on HBASE-1858:
------------------------------
Backporting Jon's fixes to TestTHLogRecover in trunk where he updates to new API seems to make it pass the test. I'm running all tests now.
> Master can't split logs created by THBase
> -----------------------------------------
>
> Key: HBASE-1858
> URL: https://issues.apache.org/jira/browse/HBASE-1858
> Project: Hadoop HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.20.0
> Reporter: Clint Morgan
> Assignee: Clint Morgan
> Fix For: 0.20.1, 0.21.0
>
> Attachments: 1858-v2.patch, 1858.patch
>
>
> When master tries to split logs created by THbase, it fails because it tries to read in the wrong key type. (THBase subclasses HLogKey to add fields to the key).
> 2009-09-16 09:03:01,943 WARN org.apache.hadoop.hbase.regionserver.HLog:
> Exception processing
> hdfs://domU-12-31-39-07-CC-A2.compute-1.internal:9000/hbase/.logs/domU-12-31-39-07-CC-A2.compute-1.internal,60020,1253103101743/hlog.dat.1253103102168
> -- continuing. Possible DATA LOSS!
> java.io.IOException: wrong key class:
> org.apache.hadoop.hbase.regionserver.HLogKey is not class
> org.apache.hadoop.hbase.regionserver.transactional.THLogKey
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1824)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:880)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:802)
> at
> org.apache.hadoop.hbase.master.ProcessServerShutdown.process(ProcessServerShutdown.java:274)
> at
> org.apache.hadoop.hbase.master.HMaster.processToDoQueue(HMaster.java:492)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:426)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1858) Master can't split logs created by
THBase
Posted by "Clint Morgan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Clint Morgan updated HBASE-1858:
--------------------------------
Attachment: 1858-v2.patch
Forgot to update the test. All tests now pass.
> Master can't split logs created by THBase
> -----------------------------------------
>
> Key: HBASE-1858
> URL: https://issues.apache.org/jira/browse/HBASE-1858
> Project: Hadoop HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.20.0
> Reporter: Clint Morgan
> Assignee: Clint Morgan
> Fix For: 0.20.1, 0.21.0
>
> Attachments: 1858-v2.patch, 1858.patch
>
>
> When master tries to split logs created by THbase, it fails because it tries to read in the wrong key type. (THBase subclasses HLogKey to add fields to the key).
> 2009-09-16 09:03:01,943 WARN org.apache.hadoop.hbase.regionserver.HLog:
> Exception processing
> hdfs://domU-12-31-39-07-CC-A2.compute-1.internal:9000/hbase/.logs/domU-12-31-39-07-CC-A2.compute-1.internal,60020,1253103101743/hlog.dat.1253103102168
> -- continuing. Possible DATA LOSS!
> java.io.IOException: wrong key class:
> org.apache.hadoop.hbase.regionserver.HLogKey is not class
> org.apache.hadoop.hbase.regionserver.transactional.THLogKey
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1824)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:880)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:802)
> at
> org.apache.hadoop.hbase.master.ProcessServerShutdown.process(ProcessServerShutdown.java:274)
> at
> org.apache.hadoop.hbase.master.HMaster.processToDoQueue(HMaster.java:492)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:426)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1858) Master can't split logs created by
THBase
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759827#action_12759827 ]
Andrew Purtell commented on HBASE-1858:
---------------------------------------
+1 will commit if all tests pass.
> Master can't split logs created by THBase
> -----------------------------------------
>
> Key: HBASE-1858
> URL: https://issues.apache.org/jira/browse/HBASE-1858
> Project: Hadoop HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.20.0
> Reporter: Clint Morgan
> Assignee: Clint Morgan
> Fix For: 0.20.1, 0.21.0
>
> Attachments: 1858-v2.patch, 1858.patch
>
>
> When master tries to split logs created by THbase, it fails because it tries to read in the wrong key type. (THBase subclasses HLogKey to add fields to the key).
> 2009-09-16 09:03:01,943 WARN org.apache.hadoop.hbase.regionserver.HLog:
> Exception processing
> hdfs://domU-12-31-39-07-CC-A2.compute-1.internal:9000/hbase/.logs/domU-12-31-39-07-CC-A2.compute-1.internal,60020,1253103101743/hlog.dat.1253103102168
> -- continuing. Possible DATA LOSS!
> java.io.IOException: wrong key class:
> org.apache.hadoop.hbase.regionserver.HLogKey is not class
> org.apache.hadoop.hbase.regionserver.transactional.THLogKey
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1824)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:880)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:802)
> at
> org.apache.hadoop.hbase.master.ProcessServerShutdown.process(ProcessServerShutdown.java:274)
> at
> org.apache.hadoop.hbase.master.HMaster.processToDoQueue(HMaster.java:492)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:426)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1858) Master can't split logs created by
THBase
Posted by "Clint Morgan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758521#action_12758521 ]
Clint Morgan commented on HBASE-1858:
-------------------------------------
I have a fix for this (define the HLogKey class in the conf), but I've found some other issues with restoring from trx log that I'm working on.
Should have something by tomorrow.
> Master can't split logs created by THBase
> -----------------------------------------
>
> Key: HBASE-1858
> URL: https://issues.apache.org/jira/browse/HBASE-1858
> Project: Hadoop HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.20.0
> Reporter: Clint Morgan
> Assignee: Clint Morgan
> Fix For: 0.20.1, 0.21.0
>
>
> When master tries to split logs created by THbase, it fails because it tries to read in the wrong key type. (THBase subclasses HLogKey to add fields to the key).
> 2009-09-16 09:03:01,943 WARN org.apache.hadoop.hbase.regionserver.HLog:
> Exception processing
> hdfs://domU-12-31-39-07-CC-A2.compute-1.internal:9000/hbase/.logs/domU-12-31-39-07-CC-A2.compute-1.internal,60020,1253103101743/hlog.dat.1253103102168
> -- continuing. Possible DATA LOSS!
> java.io.IOException: wrong key class:
> org.apache.hadoop.hbase.regionserver.HLogKey is not class
> org.apache.hadoop.hbase.regionserver.transactional.THLogKey
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1824)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:880)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:802)
> at
> org.apache.hadoop.hbase.master.ProcessServerShutdown.process(ProcessServerShutdown.java:274)
> at
> org.apache.hadoop.hbase.master.HMaster.processToDoQueue(HMaster.java:492)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:426)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1858) Master can't split logs created by
THBase
Posted by "Clint Morgan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Clint Morgan updated HBASE-1858:
--------------------------------
Attachment: 1858.patch
This patch clean up the transaction WALing, and makes it work.
- Introduce config property "hbase.regionserver.hlog.keyclass" which is used to instantiate keys.
- When we commit a transaction, we let the puts/deletes go into the normal WAL. This way, they are handled normally during recovery. The only time we need to do special WAL recovery in the transactional layer is when we find a transaction that started, but does not have an commit or abort message. In this case, its status should be in the "global" trx log.
- Clean up the use of the "global" transaction log. This holds the state of a transaction while the transaction is still in-process. This state can be forgotten after a successful commit/abort. This state is only used when we recover from the WAL and don't know what actually happened to the transaction.
- Ports HbaseBackTransactionalLogger to the new API.
- fixes a bug: when multiple puts in the same transaction to the same cell, make the last put should be used for a trx-local get.
I tested the WAL recovery and it works for me.
> Master can't split logs created by THBase
> -----------------------------------------
>
> Key: HBASE-1858
> URL: https://issues.apache.org/jira/browse/HBASE-1858
> Project: Hadoop HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.20.0
> Reporter: Clint Morgan
> Assignee: Clint Morgan
> Fix For: 0.20.1, 0.21.0
>
> Attachments: 1858.patch
>
>
> When master tries to split logs created by THbase, it fails because it tries to read in the wrong key type. (THBase subclasses HLogKey to add fields to the key).
> 2009-09-16 09:03:01,943 WARN org.apache.hadoop.hbase.regionserver.HLog:
> Exception processing
> hdfs://domU-12-31-39-07-CC-A2.compute-1.internal:9000/hbase/.logs/domU-12-31-39-07-CC-A2.compute-1.internal,60020,1253103101743/hlog.dat.1253103102168
> -- continuing. Possible DATA LOSS!
> java.io.IOException: wrong key class:
> org.apache.hadoop.hbase.regionserver.HLogKey is not class
> org.apache.hadoop.hbase.regionserver.transactional.THLogKey
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1824)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:880)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:802)
> at
> org.apache.hadoop.hbase.master.ProcessServerShutdown.process(ProcessServerShutdown.java:274)
> at
> org.apache.hadoop.hbase.master.HMaster.processToDoQueue(HMaster.java:492)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:426)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1858) Master can't split logs created by
THBase
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759847#action_12759847 ]
Andrew Purtell commented on HBASE-1858:
---------------------------------------
Committed to trunk only. TestTHLogRecovery fails for me on 0.20:
\\
{code}
Testcase: testWithoutFlush took 30.077 sec
Caused an ERROR
java.io.IOException: Only Puts in BU as of 0.20.0
java.lang.RuntimeException: java.io.Exception: Only Puts in BU as of 0.20.0
at org.apache.hadoop.hbase.client.transactional.HBaseBackedTransactionLogger.forgetTransaction
(HBaseBackedTransactionLogger.java:136)
at org.apache.hadoop.hbase.client.transactional.TransactionManager.doCommit
(TransactionManager.java:192)
at org.apache.hadoop.hbase.client.transactional.TransactionManager.tryCommit
(TransactionManager.java:154)
at org.apache.hadoop.hbase.regionserver.transactional.TestTHLogRecovery.testWithoutFlush
(TestTHLogRecovery.java:118)
{code}
> Master can't split logs created by THBase
> -----------------------------------------
>
> Key: HBASE-1858
> URL: https://issues.apache.org/jira/browse/HBASE-1858
> Project: Hadoop HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.20.0
> Reporter: Clint Morgan
> Assignee: Clint Morgan
> Fix For: 0.20.1, 0.21.0
>
> Attachments: 1858-v2.patch, 1858.patch
>
>
> When master tries to split logs created by THbase, it fails because it tries to read in the wrong key type. (THBase subclasses HLogKey to add fields to the key).
> 2009-09-16 09:03:01,943 WARN org.apache.hadoop.hbase.regionserver.HLog:
> Exception processing
> hdfs://domU-12-31-39-07-CC-A2.compute-1.internal:9000/hbase/.logs/domU-12-31-39-07-CC-A2.compute-1.internal,60020,1253103101743/hlog.dat.1253103102168
> -- continuing. Possible DATA LOSS!
> java.io.IOException: wrong key class:
> org.apache.hadoop.hbase.regionserver.HLogKey is not class
> org.apache.hadoop.hbase.regionserver.transactional.THLogKey
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1824)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:880)
> at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:802)
> at
> org.apache.hadoop.hbase.master.ProcessServerShutdown.process(ProcessServerShutdown.java:274)
> at
> org.apache.hadoop.hbase.master.HMaster.processToDoQueue(HMaster.java:492)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:426)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.