You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2010/02/02 01:56:18 UTC
[jira] Created: (HBASE-2178) Hooks for replication
Hooks for replication
---------------------
Key: HBASE-2178
URL: https://issues.apache.org/jira/browse/HBASE-2178
Project: Hadoop HBase
Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.21.0
This issue is about getting all the hooks for mdc replication in core HBase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2178) Hooks for replication
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12828406#action_12828406 ]
ryan rawson commented on HBASE-2178:
------------------------------------
+1:
fix any import package.* no wildcard imports
HBaseTestCase.java appears to be whitespace only change should be reverted
> Hooks for replication
> ---------------------
>
> Key: HBASE-2178
> URL: https://issues.apache.org/jira/browse/HBASE-2178
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.21.0
>
> Attachments: HBASE-2178.patch
>
>
> This issue is about getting all the hooks for mdc replication in core HBase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2178) Hooks for replication
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12828703#action_12828703 ]
Jean-Daniel Cryans commented on HBASE-2178:
-------------------------------------------
bq. I'm not sure I understand why you're making 4 fields public in the ZooKeeperWrapper class, the only one you're accessing in your patch for HBASE-2129 is parentZNode. And for this one I think an accessor would be better.
Agreed.
bq. In scanAddressDirectory, I would declare the List<HServerAddress> list after the check on nodes == null that does an early return. This improves readability and this way you can also invoke the constructor with the exact size of the ArrayList in advance: double win!
Since I return the list, this would not compile.
bq. In listZnodes, I don't know why you wrapped the arguments around. They fit nicely on one line.
Agreed.
bq. In writeZNode, there's some code in the try-block that's not indented.
Good catch.
bq. Just out of curiosity, why are you moving the import java.foo after the import org.apache.bar?
I will have to ask the IntelliJ authors :P
bq. Also I'm not sure I understand why you removed the call to this.conf.setBoolean("dfs.support.append", true); in setUp in TestHLog.
This is something from the 0.20 era, it should have been removed before.
> Hooks for replication
> ---------------------
>
> Key: HBASE-2178
> URL: https://issues.apache.org/jira/browse/HBASE-2178
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.21.0
>
> Attachments: HBASE-2178-2.patch, HBASE-2178.patch
>
>
> This issue is about getting all the hooks for mdc replication in core HBase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2178) Hooks for replication
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12828781#action_12828781 ]
Jean-Daniel Cryans commented on HBASE-2178:
-------------------------------------------
bq. I wonder if the interface inside SFLR shouldn't just implement http://hadoop.apache.org/common/docs/r0.19.2/api/org/apache/hadoop/fs/Seekable.html?
Yes and the SFLW should implement Syncable because at the moment it's HLog that has it and to me it doesn't make much sense.
bq. I think ZKWatcher is starting to break down. Once there was an attempt at encapsulating all ZK operations inside ZKW. Now they are starting to leak out. I think thats inevitable as zk permeates more and more of hbase.
Me too... We need a refactoring as more stuff comes in.
> Hooks for replication
> ---------------------
>
> Key: HBASE-2178
> URL: https://issues.apache.org/jira/browse/HBASE-2178
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.21.0
>
> Attachments: HBASE-2178-2.patch, HBASE-2178.patch
>
>
> This issue is about getting all the hooks for mdc replication in core HBase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2178) Hooks for replication
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12828421#action_12828421 ]
Jean-Daniel Cryans commented on HBASE-2178:
-------------------------------------------
Also I might add that this patch passes all the tests.
> Hooks for replication
> ---------------------
>
> Key: HBASE-2178
> URL: https://issues.apache.org/jira/browse/HBASE-2178
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.21.0
>
> Attachments: HBASE-2178-2.patch, HBASE-2178.patch
>
>
> This issue is about getting all the hooks for mdc replication in core HBase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2178) Hooks for replication
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12828748#action_12828748 ]
stack commented on HBASE-2178:
------------------------------
I wonder if the interface inside SFLR shouldn't just implement http://hadoop.apache.org/common/docs/r0.19.2/api/org/apache/hadoop/fs/Seekable.html?
I think ZKWatcher is starting to break down. Once there was an attempt at encapsulating all ZK operations inside ZKW. Now they are starting to leak out. I think thats inevitable as zk permeates more and more of hbase.
Otherwise, +1 (after addressing Benoit and Ryan comments)
> Hooks for replication
> ---------------------
>
> Key: HBASE-2178
> URL: https://issues.apache.org/jira/browse/HBASE-2178
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.21.0
>
> Attachments: HBASE-2178-2.patch, HBASE-2178.patch
>
>
> This issue is about getting all the hooks for mdc replication in core HBase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2178) Hooks for replication
Posted by "Benoit Sigoure (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12828497#action_12828497 ]
Benoit Sigoure commented on HBASE-2178:
---------------------------------------
I'm not sure I understand why you're making 4 fields public in the {{ZooKeeperWrapper}} class, the only one you're accessing in your patch for HBASE-2129 is {{parentZNode}}. And for this one I think an accessor would be better.
In {{scanAddressDirectory}}, I would declare the {{List<HServerAddress> list}} after the check on {{nodes == null}} that does an early return. This improves readability and this way you can also invoke the constructor with the exact size of the {{ArrayList}} in advance: double win!
In {{listZnodes}}, I don't know why you wrapped the arguments around. They fit nicely on one line.
In {{writeZNode}}, there's some code in the try-block that's not indented.
Just out of curiosity, why are you moving the {{import java.foo}} after the {{import org.apache.bar}}?
Also I'm not sure I understand why you removed the call to {{this.conf.setBoolean("dfs.support.append", true);}} in {{setUp}} in {{TestHLog}}.
> Hooks for replication
> ---------------------
>
> Key: HBASE-2178
> URL: https://issues.apache.org/jira/browse/HBASE-2178
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.21.0
>
> Attachments: HBASE-2178-2.patch, HBASE-2178.patch
>
>
> This issue is about getting all the hooks for mdc replication in core HBase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2178) Hooks for replication
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-2178:
--------------------------------------
Attachment: HBASE-2178.patch
This patch is basically the src/java part of HBASE-2129. List of changes:
# HConstants
New constant for the znode parent.
# HConnectionManager
Introduces a new way of keeping tab of the ZK quorums so that a single HBase client can talk to 2 different HBase clusters managed by the same quorum.
# HbaseObjectWritable
Adds new classes to the Map.
# HRegion
Now passed the whole HRegionInfo when appending.
# HRegionServer
join() is down protected so child classes can override it and adds visibility to threadWakeFrequency
# HLog
The reader now offers seek and getPosition, good for reusability.
doWrite now takes the regionInfo instead of just the name.
HLog.Entry now implements Writable
# SequenceFileLogReader
Implements the 2 new methods.
# SequenceFileLogWriter
Calls hflush instead of sync.
# ZooKeeperWrapper
Mostly refactoring and new facilities.
# HBaseTestingUtility
Makes the conf settable in the constructor.
Refactors the code to start a mini ZK cluster.
Provides a way to truncate a table very fast.
# TestStoreReconstruction and TestHLog
Refactorings related to HLog modifications.
> Hooks for replication
> ---------------------
>
> Key: HBASE-2178
> URL: https://issues.apache.org/jira/browse/HBASE-2178
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.21.0
>
> Attachments: HBASE-2178.patch
>
>
> This issue is about getting all the hooks for mdc replication in core HBase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2178) Hooks for replication
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12828795#action_12828795 ]
Jean-Daniel Cryans commented on HBASE-2178:
-------------------------------------------
But the problem of having the Reader and Writer implementing those interfaces makes it that it may not make sense for other implementations like Avro?
> Hooks for replication
> ---------------------
>
> Key: HBASE-2178
> URL: https://issues.apache.org/jira/browse/HBASE-2178
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.21.0
>
> Attachments: HBASE-2178-2.patch, HBASE-2178.patch
>
>
> This issue is about getting all the hooks for mdc replication in core HBase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2178) Hooks for replication
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-2178:
--------------------------------------
Attachment: HBASE-2178-2.patch
New version of patch that fixes some nits Ryan found, the comments on the previous patch still all apply.
> Hooks for replication
> ---------------------
>
> Key: HBASE-2178
> URL: https://issues.apache.org/jira/browse/HBASE-2178
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.21.0
>
> Attachments: HBASE-2178-2.patch, HBASE-2178.patch
>
>
> This issue is about getting all the hooks for mdc replication in core HBase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-2178) Hooks for replication
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans resolved HBASE-2178.
---------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Thank you guys for the comments and reviews, I committed the patch with changes to trunk.
> Hooks for replication
> ---------------------
>
> Key: HBASE-2178
> URL: https://issues.apache.org/jira/browse/HBASE-2178
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.21.0
>
> Attachments: HBASE-2178-2.patch, HBASE-2178.patch
>
>
> This issue is about getting all the hooks for mdc replication in core HBase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.