You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Jonathan Gray <jg...@apache.org> on 2010/07/24 01:25:29 UTC
Review Request: ZKAssign. New zookeeper assignment methods.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/382/
-----------------------------------------------------------
Review request for hbase, stack, Jean-Daniel Cryans, Karthik Ranganathan, and Kannan Muthukkaruppan.
Summary
-------
This method implements all the zookeeper methods needed for this last part of the doc posted in HBASE-2692 (definition of valid creation/transitions for unassigned znodes):
MASTER
1. Master creates an unassigned node as OFFLINE.
- Cluster startup and table enabling.
2. Master forces an existing unassigned node to OFFLINE.
- RegionServer failure.
- Allows transitions from all states to OFFLINE.
3. Master deletes an unassigned node that was in a OPENED state.
- Normal region transitions. Besides cluster startup, no other deletions
of unassigned nodes is allowed.
4. Master deletes all unassigned nodes regardless of state.
- Cluster startup before any assignment happens.
REGIONSERVER
1. RegionServer creates an unassigned node as CLOSING.
- All region closes will do this in response to a CLOSE RPC from Master.
- A node can never be transitioned to CLOSING, only created.
2. RegionServer transitions an unassigned node from CLOSING to CLOSED.
- Normal region closes. CAS operation.
3. RegionServer transitions an unassigned node from OFFLINE to OPENING.
- All region opens will do this in response to an OPEN RPC from the Master.
- Normal region opens. CAS operation.
4. RegionServer transitions an unassigned node from OPENING to OPENED.
- Normal region opens. CAS operation.
This addresses bug HBASE-2697.
http://issues.apache.org/jira/browse/HBASE-2697
Diffs
-----
branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java PRE-CREATION
Diff: http://review.hbase.org/r/382/diff
Testing
-------
Unit test to come. Just putting up since there have been some recent issues with multiple transitions and stuff, wanted to show what the new stuff looks like.
Thanks,
Jonathan
Re: Review Request: ZKAssign. New zookeeper assignment methods.
Posted by st...@duboce.net.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/382/#review498
-----------------------------------------------------------
branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
<http://review.hbase.org/r/382/#comment2019>
Put your big doc up here in class comment? Or you want to make a document of it? Or you want me to do it for you?
branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
<http://review.hbase.org/r/382/#comment2016>
The way this method is named would seem to imply it does something internally that has to do w/ 'offlining' but all I see is the createAndWatch and passing of whatever is in data.getBytes(). It makes me think this method is not properly scoped -- that it should be doing more?
branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
<http://review.hbase.org/r/382/#comment2017>
Great javadoc.
branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
<http://review.hbase.org/r/382/#comment2018>
I have the same issue w/ this method. You are passing in the 'generic' RegionTransitionData type which seems like it could be anything?
branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
<http://review.hbase.org/r/382/#comment2020>
This method is same as createNodeOffline but for the log message. As said above, it makes me think the method should be broader
branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
<http://review.hbase.org/r/382/#comment2021>
Whats in the 'data' passed in here?
- stack
On 2010-07-23 16:29:43, Jonathan Gray wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.hbase.org/r/382/
> -----------------------------------------------------------
>
> (Updated 2010-07-23 16:29:43)
>
>
> Review request for hbase, stack, Jean-Daniel Cryans, Karthik Ranganathan, and Kannan Muthukkaruppan.
>
>
> Summary
> -------
>
> This method implements all the zookeeper methods needed for this last part of the doc posted in HBASE-2692 (definition of valid creation/transitions for unassigned znodes):
>
>
> MASTER
>
> 1. Master creates an unassigned node as OFFLINE.
>
> - Cluster startup and table enabling.
>
> 2. Master forces an existing unassigned node to OFFLINE.
>
> - RegionServer failure.
>
> - Allows transitions from all states to OFFLINE.
>
> 3. Master deletes an unassigned node that was in a OPENED state.
>
> - Normal region transitions. Besides cluster startup, no other deletions
> of unassigned nodes is allowed.
>
> 4. Master deletes all unassigned nodes regardless of state.
>
> - Cluster startup before any assignment happens.
>
> REGIONSERVER
>
> 1. RegionServer creates an unassigned node as CLOSING.
>
> - All region closes will do this in response to a CLOSE RPC from Master.
>
> - A node can never be transitioned to CLOSING, only created.
>
> 2. RegionServer transitions an unassigned node from CLOSING to CLOSED.
>
> - Normal region closes. CAS operation.
>
> 3. RegionServer transitions an unassigned node from OFFLINE to OPENING.
>
> - All region opens will do this in response to an OPEN RPC from the Master.
>
> - Normal region opens. CAS operation.
>
> 4. RegionServer transitions an unassigned node from OPENING to OPENED.
>
> - Normal region opens. CAS operation.
>
>
> Should I put above into the class comment javadoc or is it too much for there?
>
>
> This addresses bug HBASE-2697.
> http://issues.apache.org/jira/browse/HBASE-2697
>
>
> Diffs
> -----
>
> branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java PRE-CREATION
>
> Diff: http://review.hbase.org/r/382/diff
>
>
> Testing
> -------
>
> Unit test to come. Just putting up since there have been some recent issues with multiple transitions and stuff, wanted to show what the new stuff looks like.
>
>
> Thanks,
>
> Jonathan
>
>
Re: Review Request: ZKAssign. New zookeeper assignment methods.
Posted by Jonathan Gray <jg...@apache.org>.
> On 2010-07-23 16:35:42, Jonathan Gray wrote:
> > Thinking on it, I will probably need to not just blindly force something to OFFLINE in the masters force to OFFLINE. Though any state will be able to be transitioned to OFFLINE (during recovery, we may need to restart the assignment of something, setting to OFFLINE first is what allows us to timeout if for some reason an RS doesn't respond/act on an OPEN rpc), we will want to use more of a CAS operation so we don't accidentally miss a transition on that node.
> >
> > For example, we may timeout an OFFLINE because it took too long, so we want to switch it back to a new version of OFFLINE and assign to someone else. We'll want to ensure when setting to OFFLINE it's still in the previously seen OFFLINE version, otherwise a server may have started OPENING in between our check and set.
(this is not totally clear to me yet, going to wait until i get a bit further in integration of this class)
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/382/#review476
-----------------------------------------------------------
On 2010-07-23 16:29:43, Jonathan Gray wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.hbase.org/r/382/
> -----------------------------------------------------------
>
> (Updated 2010-07-23 16:29:43)
>
>
> Review request for hbase, stack, Jean-Daniel Cryans, Karthik Ranganathan, and Kannan Muthukkaruppan.
>
>
> Summary
> -------
>
> This method implements all the zookeeper methods needed for this last part of the doc posted in HBASE-2692 (definition of valid creation/transitions for unassigned znodes):
>
>
> MASTER
>
> 1. Master creates an unassigned node as OFFLINE.
>
> - Cluster startup and table enabling.
>
> 2. Master forces an existing unassigned node to OFFLINE.
>
> - RegionServer failure.
>
> - Allows transitions from all states to OFFLINE.
>
> 3. Master deletes an unassigned node that was in a OPENED state.
>
> - Normal region transitions. Besides cluster startup, no other deletions
> of unassigned nodes is allowed.
>
> 4. Master deletes all unassigned nodes regardless of state.
>
> - Cluster startup before any assignment happens.
>
> REGIONSERVER
>
> 1. RegionServer creates an unassigned node as CLOSING.
>
> - All region closes will do this in response to a CLOSE RPC from Master.
>
> - A node can never be transitioned to CLOSING, only created.
>
> 2. RegionServer transitions an unassigned node from CLOSING to CLOSED.
>
> - Normal region closes. CAS operation.
>
> 3. RegionServer transitions an unassigned node from OFFLINE to OPENING.
>
> - All region opens will do this in response to an OPEN RPC from the Master.
>
> - Normal region opens. CAS operation.
>
> 4. RegionServer transitions an unassigned node from OPENING to OPENED.
>
> - Normal region opens. CAS operation.
>
>
> Should I put above into the class comment javadoc or is it too much for there?
>
>
> This addresses bug HBASE-2697.
> http://issues.apache.org/jira/browse/HBASE-2697
>
>
> Diffs
> -----
>
> branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java PRE-CREATION
>
> Diff: http://review.hbase.org/r/382/diff
>
>
> Testing
> -------
>
> Unit test to come. Just putting up since there have been some recent issues with multiple transitions and stuff, wanted to show what the new stuff looks like.
>
>
> Thanks,
>
> Jonathan
>
>
Re: Review Request: ZKAssign. New zookeeper assignment methods.
Posted by Jonathan Gray <jg...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/382/#review476
-----------------------------------------------------------
Thinking on it, I will probably need to not just blindly force something to OFFLINE in the masters force to OFFLINE. Though any state will be able to be transitioned to OFFLINE (during recovery, we may need to restart the assignment of something, setting to OFFLINE first is what allows us to timeout if for some reason an RS doesn't respond/act on an OPEN rpc), we will want to use more of a CAS operation so we don't accidentally miss a transition on that node.
For example, we may timeout an OFFLINE because it took too long, so we want to switch it back to a new version of OFFLINE and assign to someone else. We'll want to ensure when setting to OFFLINE it's still in the previously seen OFFLINE version, otherwise a server may have started OPENING in between our check and set.
- Jonathan
On 2010-07-23 16:29:43, Jonathan Gray wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.hbase.org/r/382/
> -----------------------------------------------------------
>
> (Updated 2010-07-23 16:29:43)
>
>
> Review request for hbase, stack, Jean-Daniel Cryans, Karthik Ranganathan, and Kannan Muthukkaruppan.
>
>
> Summary
> -------
>
> This method implements all the zookeeper methods needed for this last part of the doc posted in HBASE-2692 (definition of valid creation/transitions for unassigned znodes):
>
>
> MASTER
>
> 1. Master creates an unassigned node as OFFLINE.
>
> - Cluster startup and table enabling.
>
> 2. Master forces an existing unassigned node to OFFLINE.
>
> - RegionServer failure.
>
> - Allows transitions from all states to OFFLINE.
>
> 3. Master deletes an unassigned node that was in a OPENED state.
>
> - Normal region transitions. Besides cluster startup, no other deletions
> of unassigned nodes is allowed.
>
> 4. Master deletes all unassigned nodes regardless of state.
>
> - Cluster startup before any assignment happens.
>
> REGIONSERVER
>
> 1. RegionServer creates an unassigned node as CLOSING.
>
> - All region closes will do this in response to a CLOSE RPC from Master.
>
> - A node can never be transitioned to CLOSING, only created.
>
> 2. RegionServer transitions an unassigned node from CLOSING to CLOSED.
>
> - Normal region closes. CAS operation.
>
> 3. RegionServer transitions an unassigned node from OFFLINE to OPENING.
>
> - All region opens will do this in response to an OPEN RPC from the Master.
>
> - Normal region opens. CAS operation.
>
> 4. RegionServer transitions an unassigned node from OPENING to OPENED.
>
> - Normal region opens. CAS operation.
>
>
> Should I put above into the class comment javadoc or is it too much for there?
>
>
> This addresses bug HBASE-2697.
> http://issues.apache.org/jira/browse/HBASE-2697
>
>
> Diffs
> -----
>
> branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java PRE-CREATION
>
> Diff: http://review.hbase.org/r/382/diff
>
>
> Testing
> -------
>
> Unit test to come. Just putting up since there have been some recent issues with multiple transitions and stuff, wanted to show what the new stuff looks like.
>
>
> Thanks,
>
> Jonathan
>
>
Re: Review Request: ZKAssign. New zookeeper assignment methods.
Posted by Jonathan Gray <jg...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/382/
-----------------------------------------------------------
(Updated 2010-07-23 16:29:43.251978)
Review request for hbase, stack, Jean-Daniel Cryans, Karthik Ranganathan, and Kannan Muthukkaruppan.
Changes
-------
Should I put above description into the class comment javadoc or is it too much for there?
Summary (updated)
-------
This method implements all the zookeeper methods needed for this last part of the doc posted in HBASE-2692 (definition of valid creation/transitions for unassigned znodes):
MASTER
1. Master creates an unassigned node as OFFLINE.
- Cluster startup and table enabling.
2. Master forces an existing unassigned node to OFFLINE.
- RegionServer failure.
- Allows transitions from all states to OFFLINE.
3. Master deletes an unassigned node that was in a OPENED state.
- Normal region transitions. Besides cluster startup, no other deletions
of unassigned nodes is allowed.
4. Master deletes all unassigned nodes regardless of state.
- Cluster startup before any assignment happens.
REGIONSERVER
1. RegionServer creates an unassigned node as CLOSING.
- All region closes will do this in response to a CLOSE RPC from Master.
- A node can never be transitioned to CLOSING, only created.
2. RegionServer transitions an unassigned node from CLOSING to CLOSED.
- Normal region closes. CAS operation.
3. RegionServer transitions an unassigned node from OFFLINE to OPENING.
- All region opens will do this in response to an OPEN RPC from the Master.
- Normal region opens. CAS operation.
4. RegionServer transitions an unassigned node from OPENING to OPENED.
- Normal region opens. CAS operation.
Should I put above into the class comment javadoc or is it too much for there?
This addresses bug HBASE-2697.
http://issues.apache.org/jira/browse/HBASE-2697
Diffs
-----
branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java PRE-CREATION
Diff: http://review.hbase.org/r/382/diff
Testing
-------
Unit test to come. Just putting up since there have been some recent issues with multiple transitions and stuff, wanted to show what the new stuff looks like.
Thanks,
Jonathan
Re: Review Request: ZKAssign. New zookeeper assignment methods.
Posted by Jonathan Gray <jg...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/382/
-----------------------------------------------------------
(Updated 2010-07-23 16:28:15.057431)
Review request for hbase, stack, Jean-Daniel Cryans, Karthik Ranganathan, and Kannan Muthukkaruppan.
Changes
-------
Whitespace take 2
Summary
-------
This method implements all the zookeeper methods needed for this last part of the doc posted in HBASE-2692 (definition of valid creation/transitions for unassigned znodes):
MASTER
1. Master creates an unassigned node as OFFLINE.
- Cluster startup and table enabling.
2. Master forces an existing unassigned node to OFFLINE.
- RegionServer failure.
- Allows transitions from all states to OFFLINE.
3. Master deletes an unassigned node that was in a OPENED state.
- Normal region transitions. Besides cluster startup, no other deletions
of unassigned nodes is allowed.
4. Master deletes all unassigned nodes regardless of state.
- Cluster startup before any assignment happens.
REGIONSERVER
1. RegionServer creates an unassigned node as CLOSING.
- All region closes will do this in response to a CLOSE RPC from Master.
- A node can never be transitioned to CLOSING, only created.
2. RegionServer transitions an unassigned node from CLOSING to CLOSED.
- Normal region closes. CAS operation.
3. RegionServer transitions an unassigned node from OFFLINE to OPENING.
- All region opens will do this in response to an OPEN RPC from the Master.
- Normal region opens. CAS operation.
4. RegionServer transitions an unassigned node from OPENING to OPENED.
- Normal region opens. CAS operation.
This addresses bug HBASE-2697.
http://issues.apache.org/jira/browse/HBASE-2697
Diffs (updated)
-----
branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java PRE-CREATION
Diff: http://review.hbase.org/r/382/diff
Testing
-------
Unit test to come. Just putting up since there have been some recent issues with multiple transitions and stuff, wanted to show what the new stuff looks like.
Thanks,
Jonathan
Re: Review Request: ZKAssign. New zookeeper assignment methods.
Posted by Jonathan Gray <jg...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/382/
-----------------------------------------------------------
(Updated 2010-07-23 16:26:26.834022)
Review request for hbase, stack, Jean-Daniel Cryans, Karthik Ranganathan, and Kannan Muthukkaruppan.
Changes
-------
Whitespace cleanup
Summary
-------
This method implements all the zookeeper methods needed for this last part of the doc posted in HBASE-2692 (definition of valid creation/transitions for unassigned znodes):
MASTER
1. Master creates an unassigned node as OFFLINE.
- Cluster startup and table enabling.
2. Master forces an existing unassigned node to OFFLINE.
- RegionServer failure.
- Allows transitions from all states to OFFLINE.
3. Master deletes an unassigned node that was in a OPENED state.
- Normal region transitions. Besides cluster startup, no other deletions
of unassigned nodes is allowed.
4. Master deletes all unassigned nodes regardless of state.
- Cluster startup before any assignment happens.
REGIONSERVER
1. RegionServer creates an unassigned node as CLOSING.
- All region closes will do this in response to a CLOSE RPC from Master.
- A node can never be transitioned to CLOSING, only created.
2. RegionServer transitions an unassigned node from CLOSING to CLOSED.
- Normal region closes. CAS operation.
3. RegionServer transitions an unassigned node from OFFLINE to OPENING.
- All region opens will do this in response to an OPEN RPC from the Master.
- Normal region opens. CAS operation.
4. RegionServer transitions an unassigned node from OPENING to OPENED.
- Normal region opens. CAS operation.
This addresses bug HBASE-2697.
http://issues.apache.org/jira/browse/HBASE-2697
Diffs (updated)
-----
branches/0.90_master_rewrite/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java PRE-CREATION
Diff: http://review.hbase.org/r/382/diff
Testing
-------
Unit test to come. Just putting up since there have been some recent issues with multiple transitions and stuff, wanted to show what the new stuff looks like.
Thanks,
Jonathan