You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Rakesh R (JIRA)" <ji...@apache.org> on 2012/07/17 11:05:34 UTC
[jira] [Created] (BOOKKEEPER-347) Provide mechanism to detect r-o
bookie by the bookie clients
Rakesh R created BOOKKEEPER-347:
-----------------------------------
Summary: Provide mechanism to detect r-o bookie by the bookie clients
Key: BOOKKEEPER-347
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-client
Affects Versions: 4.0.0
Reporter: Rakesh R
Assignee: Rakesh R
This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507468#comment-13507468 ]
Ivan Kelly commented on BOOKKEEPER-347:
---------------------------------------
New patch looks good, but patches are failing so you should look into that. Also, there's a findbugs and 1 line which is too long.
Also, could you rename BKInvalidWriteOnReadOnlyBookieException to BKWriteToReadOnlyBookieException (and the other objects in that file named the same also).
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497817#comment-13497817 ]
Vinay commented on BOOKKEEPER-347:
----------------------------------
Hi Ivan,
{quote}The problem is that, in BookieWatcher, you remove the READONLY znode, which only removes "readonly" from the list, not the znodes under "readonly"{quote}
Here, everytime getChildren() is called, new list of children itself will be used as knownbookies. This list contains only one level of znodes. i.e. It doesn't contains the znodes under 'readonly', but it contains 'readonly'.
Since, during transition to readOnly, we will unregister from 'available' and register under 'readonly' znode, getChildren(..) call will not have this transitioned bookie address. So this bookie will be removed automatically from available bookies on watcher event.
Only thing we need to do is, to avoid closing of the readonly bookie clients, we can add watcher on 'readonly' znode and keep track of readonly bookies, and avoid closing of clients for readonly bookies.
In the above Test,
I was trying to simulate the transitionToReadOnly. Anyway I corrected the test by directly using the {{Bookie.transitionToReadOnly()}}.
One small change required in {{Bookie.transitionToReadOnly()}} is the order of creating the readonly ephemeral node and deleting the available ephemeral node should be changed to get the updated list of readonly bookies first.
I will post a patch soon regarding these..
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-347) Provide mechanism to detect r-o
bookie by the bookie clients
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay updated BOOKKEEPER-347:
-----------------------------
Attachment: BOOKKEEPER-347.patch
Attaching the patch
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-347) Provide mechanism to detect r-o
bookie by the bookie clients
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay updated BOOKKEEPER-347:
-----------------------------
Attachment: BOOKKEEPER-347.patch
Attaching the patch with fix
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508839#comment-13508839 ]
Hadoop QA commented on BOOKKEEPER-347:
--------------------------------------
Testing JIRA BOOKKEEPER-347
WARNING: Running test-patch on a dirty local svn workspace
Patch <a href="/jira/secure/attachment/12555777/BOOKKEEPER-347.patch">/jira/secure/attachment/12555777/BOOKKEEPER-347.patch</a> downloaded at Mon Dec 3 16:11:39 UTC 2012
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any line longer than 120
. {color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1 FINDBUGS{color}
. {color:green}+1{color} the patch does not seem to introduce new Findbugs warnings
{color:green}+1 TESTS{color}
. Tests run: 387
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/bookkeeper-trunk-precommit-build/69/
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (BOOKKEEPER-347) Provide mechanism to
detect r-o bookie by the bookie clients
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507468#comment-13507468 ]
Ivan Kelly edited comment on BOOKKEEPER-347 at 11/30/12 5:21 PM:
-----------------------------------------------------------------
New patch looks good, but tests are failing so you should look into that. Also, there's a findbugs and 1 line which is too long.
Also, could you rename BKInvalidWriteOnReadOnlyBookieException to BKWriteToReadOnlyBookieException (and the other objects in that file named the same also).
was (Author: ikelly):
New patch looks good, but patches are failing so you should look into that. Also, there's a findbugs and 1 line which is too long.
Also, could you rename BKInvalidWriteOnReadOnlyBookieException to BKWriteToReadOnlyBookieException (and the other objects in that file named the same also).
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487193#comment-13487193 ]
Ivan Kelly commented on BOOKKEEPER-347:
---------------------------------------
testLedgerCreationShouldFailWithReadonlyBookie() is broken. Try replacing the 2 zk lines with a 3 second thread sleep. it still passes. you've killed one of the two bookies at the start of the test, so there's no way you'll be able to create a ledger.
The problem is that, in BookieWatcher, you remove the READONLY znode, which only removes "readonly" from the list, not the znodes under "readonly". Really, what you need to do is watch /available/readonly similar to how /available is watched, and update a list of readonly bookies when the children change. this list can then be remove for the desired effect.
Finally, in the test, you should use BookKeeperClusterTestCase#getBookie() instead of hard coding the bookie id.
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Rakesh R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485569#comment-13485569 ]
Rakesh R commented on BOOKKEEPER-347:
-------------------------------------
Adding more to my above ??comment#1??:
We could also think of an optimistic approach of sending the write request to bkserver(without extra r-o checks), anyway bookie server has the logic of validating r-o mode and throws BookieProtocol.EREADONLY exception back to the client. So in the client side, just handle "BookieProtocol.EREADONLY" and finds new bookie server to continue writing entries.
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508637#comment-13508637 ]
Ivan Kelly commented on BOOKKEEPER-347:
---------------------------------------
testLedgerCreationShouldFailWithReadonlyBookie is still flakey. This is due to the bookieWatcher in the bookie client not being updated immediately when the bookie transitions. It has to wait for the watcher to trigger, but by this time, createLedger may have already been called. This can be solved by modifying the test to the following
{code}
/**
* Test ledger creation with readonly bookies
*/
public void testLedgerCreationShouldFailWithReadonlyBookie() throws Exception {
killBookie(1);
baseConf.setReadOnlyModeEnabled(true);
startNewBookie();
bs.get(1).getBookie().transitionToReadOnlyMode();
try {
bkc.readBookiesBlocking();
bkc.createLedger(2, 2, DigestType.CRC32, "".getBytes());
fail("Must throw exception, as there is one readonly bookie");
} catch (BKException e) {
// Expected
}
}
{code}
I also rejigged the how you start the bookie. I don't like changing the conf of the bookie on the fly, as the result on the bookie side could be unpredictable.
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509600#comment-13509600 ]
Hudson commented on BOOKKEEPER-347:
-----------------------------------
Integrated in bookkeeper-trunk #837 (See [https://builds.apache.org/job/bookkeeper-trunk/837/])
BOOKKEEPER-347: Provide mechanism to detect r-o bookie by the bookie clients (Vinay via ivank) (Revision 1416590)
Result = UNSTABLE
ivank :
Files :
* /zookeeper/bookkeeper/trunk/CHANGES.txt
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookieWatcher.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/ReadOnlyBookieTest.java
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508596#comment-13508596 ]
Hadoop QA commented on BOOKKEEPER-347:
--------------------------------------
Testing JIRA BOOKKEEPER-347
WARNING: Running test-patch on a dirty local svn workspace
Patch <a href="/jira/secure/attachment/12555711/BOOKKEEPER-347.patch">/jira/secure/attachment/12555711/BOOKKEEPER-347.patch</a> downloaded at Mon Dec 3 08:41:01 UTC 2012
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any line longer than 120
. {color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1 FINDBUGS{color}
. {color:green}+1{color} the patch does not seem to introduce new Findbugs warnings
{color:red}-1 TESTS{color}
. Tests run: 387
. Tests failed: 1
. Tests errors: 0
. The patch failed the following testcases:
. testLedgerCreationShouldFailWithReadonlyBookie(org.apache.bookkeeper.test.ReadOnlyBookieTest)
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/bookkeeper-trunk-precommit-build/63/
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485907#comment-13485907 ]
Uma Maheswara Rao G commented on BOOKKEEPER-347:
------------------------------------------------
{code}
for (String bookieAddrString : children) {
+ if (Bookie.READONLY.equals(bookieAddrString)) {
+ // Just exclude the readonly bookies from available nodes
+ // list.
+ continue;
+ }
{code}
I think there is a potential issue here.
When you simply exclude the r-o bookie from newBookieAddrs set, we will simply consider the remaining bookies as dead Bookies and will close all the connections. Closing connections for writers may be ok, but this is not at all correct for readers as we will allow reading the r-o bookies.
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508676#comment-13508676 ]
Vinay commented on BOOKKEEPER-347:
----------------------------------
Ok, thanks for the suggestion Ivan. I will modify and upload the patch again.
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Rakesh R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485562#comment-13485562 ]
Rakesh R commented on BOOKKEEPER-347:
-------------------------------------
Thanks Vinay and nice work. Just following comments:
----
comment#1:
In your patch, its iterating over bookieAddresses and comparing with 'readonly' string. Instead we could check the corresponding bookie address present in '{bookieRegPath}/readonly/' znode. isn't it?
{code}
HashSet<InetSocketAddress> newBookieAddrs = new HashSet<InetSocketAddress>();
for (String bookieAddrString : children) {
+ if (Bookie.READONLY.equals(bookieAddrString)) {
+ // Just exclude the readonly bookies from available nodes
+ // list.
+ continue;
+ }
{code}
----
comment#2:
Also, its good to have test cases verifying the behaviour, like we can just creating an ephemeral znode bookieip:port under /readonly and try creating ledger using bkclient, should through exception.
-Rakesh
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-347) Provide mechanism to detect r-o
bookie by the bookie clients
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay updated BOOKKEEPER-347:
-----------------------------
Attachment: BOOKKEEPER-347.patch
Attaching patch for client side changes.
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-347) Provide mechanism to detect r-o
bookie by the bookie clients
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay updated BOOKKEEPER-347:
-----------------------------
Attachment: BOOKKEEPER-347.patch
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Rakesh R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485887#comment-13485887 ]
Rakesh R commented on BOOKKEEPER-347:
-------------------------------------
@Vinay
I understood. I got confused after reading the comments in the patch.
{code}
+ // Just exclude the readonly bookies from available nodes
+ // list.
{code}
Instead could you please mention like:
"// Excluding 'readonly' znode from the available bookies list"
Also, we can directly remove the Bookie.READONLY as follows, in that case not required to have the checks inside loop:
{code}
children.remove(Bookie.READONLY);
{code}
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485832#comment-13485832 ]
Vinay commented on BOOKKEEPER-347:
----------------------------------
Hi Rakesh,
"readonly" is the persistent znode, which is under "/ledgers/available" and if any r-o bookies will be having ephemeral nodes under "/ledgers/available/readonly".
As Sijie mentioned in https://issues.apache.org/jira/browse/BOOKKEEPER-199?focusedCommentId=13416144&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13416144 only write requests will go through available bookies and we need exclude r-o bookies from available bookies list. Here since "readonly" znode also under "/ledgers/available", Just excluding it.
Since BookieWatcher itself will exclude from available list, no need of extra checking of whether bookie is readonly or not at client side.
Requests, which have sent before BookieWatcher updates the available list, will get "BookieProtocol.EREADONLY" error. And again ensemble change will happen excluding the r-o bookies.
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504565#comment-13504565 ]
Hadoop QA commented on BOOKKEEPER-347:
--------------------------------------
Testing JIRA BOOKKEEPER-347
WARNING: Running test-patch on a dirty local svn workspace
Patch <a href="/jira/secure/attachment/12553635/BOOKKEEPER-347.patch">/jira/secure/attachment/12553635/BOOKKEEPER-347.patch</a> downloaded at Tue Nov 27 11:53:06 UTC 2012
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:red}-1{color} the patch contains 1 line(s) longer than 120 characters
. {color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
. {color:red}WARNING{color}: the current HEAD has 8 Javadoc warning(s)
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
. {color:red}WARNING{color}: the current HEAD has 9 javac warning(s)
{color:green}+1 FINDBUGS{color}
. {color:green}+1{color} the patch does not seem to introduce new Findbugs warnings
. {color:red}WARNING: the current HEAD has Findbugs warning(s), they should be addressed ASAP{color}
{color:red}-1 TESTS{color}
. Tests run: 390
. Tests failed: 5
. Tests errors: 36
. The patch failed the following testcases:
. testLedgerCreationShouldFailWithReadonlyBookie(org.apache.bookkeeper.test.ReadOnlyBookieTest)
. testAsyncBookieRecoveryToRandomBookies[0](org.apache.bookkeeper.client.BookieRecoveryTest)
. testAsyncBookieRecoveryToRandomBookies[1](org.apache.bookkeeper.client.BookieRecoveryTest)
. testAsyncBookieRecoveryToRandomBookies[2](org.apache.bookkeeper.client.BookieRecoveryTest)
. testAsyncBookieRecoveryToRandomBookies[3](org.apache.bookkeeper.client.BookieRecoveryTest)
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/bookkeeper-trunk-precommit-build/31/
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-347) Provide mechanism to detect r-o
bookie by the bookie clients
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay updated BOOKKEEPER-347:
-----------------------------
Attachment: BOOKKEEPER-347.patch
Attaching the latest patchn with test failure fix
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-347) Provide mechanism to detect
r-o bookie by the bookie clients
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485575#comment-13485575 ]
Uma Maheswara Rao G commented on BOOKKEEPER-347:
------------------------------------------------
Thanks a lot Vinay for the patch!
I also wanted to review this change. Currently I am not able to look at this patch as I am on vacation. I will be back by tomorrow. Overall Nice work from you in BK-199 JIRA :-).
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-347) Provide mechanism to detect r-o
bookie by the bookie clients
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay updated BOOKKEEPER-347:
-----------------------------
Attachment: BOOKKEEPER-347.patch
Attaching the latest patch.
> Provide mechanism to detect r-o bookie by the bookie clients
> -------------------------------------------------------------
>
> Key: BOOKKEEPER-347
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-347
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client
> Affects Versions: 4.1.0
> Reporter: Rakesh R
> Assignee: Vinay
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-347.patch, BOOKKEEPER-347.patch
>
>
> This jira to discuss, how the bookie client knows about the bookie running in r-o. This would be required by the client to choose writable bookies during add entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira