You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Sijie Guo (Created) (JIRA)" <ji...@apache.org> on 2012/02/07 12:52:59 UTC
[jira] [Created] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Prevent incorrect NoSuchLedgerException for readLastConfirmed.
--------------------------------------------------------------
Key: BOOKKEEPER-163
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
Project: Bookkeeper
Issue Type: Bug
Components: bookkeeper-client, bookkeeper-server
Affects Versions: 4.0.0
Reporter: Sijie Guo
bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
1) A disk is replaced or ledger index is removed by a sloppy admin.
2) A disk is not mounted when a bookie machine is restarted.
We need a mechanism to prevent such incorrect responses.
Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228247#comment-13228247 ]
Sijie Guo commented on BOOKKEEPER-163:
--------------------------------------
committed as r1299984. Great work, Thanks Ivan.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227332#comment-13227332 ]
Sijie Guo commented on BOOKKEEPER-163:
--------------------------------------
new patch is well done. but a new issue came to me when I reviewed this patch. I am not sure that copying files is good enough for upgrading, since it would take long time to copy data and exhaust disk space (if we don't have enough space to copy data, the upgrading would be failed). do you consider creating hardlink instead of copying data for upgrading?
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225991#comment-13225991 ]
Sijie Guo commented on BOOKKEEPER-163:
--------------------------------------
> I did have this in the back of my mind when writing this, but it seems an extra feature to me and would bloat the scope a bit much. Could we put this in an new JIRA?
yeah, it is an extra feature. let's add it in a new jira.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225975#comment-13225975 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-163:
----------------------------------------------------------
bq. On 2012-03-09 06:38:44, Sijie Guo wrote:
bq. > bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CookieTest.java, line 151
bq. > <https://reviews.apache.org/r/3926/diff/1/?file=75432#file75432line151>
bq. >
bq. > for a new disk (new directory) added case, it would not cause data missing.
bq. >
bq. > could we provide a tool to upgrade a bookie's cookie when adding a new directory?
I did have this in the back of my mind when writing this, but it seems an extra feature to me and would bloat the scope a bit much. Could we put this in an new JIRA?
bq. On 2012-03-09 06:38:44, Sijie Guo wrote:
bq. > bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java, line 173
bq. > <https://reviews.apache.org/r/3926/diff/1/?file=75428#file75428line173>
bq. >
bq. > it would be better to copy data to a temp directory, such as upgrading, then rename the upgrading directory to current.
bq. >
bq. > otherwise, if the upgrading is broken after created current directory, but user restarts bookie server with this upgrade-failed directory, bookie server would treat it as a new environment.
Very good point, will do this.
bq. On 2012-03-09 06:38:44, Sijie Guo wrote:
bq. > bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java, line 63
bq. > <https://reviews.apache.org/r/3926/diff/1/?file=75428#file75428line63>
bq. >
bq. > it seems that you missed that file 'lastId' and 'lastMark'
Ah, well spotted, will add them.
bq. On 2012-03-09 06:38:44, Sijie Guo wrote:
bq. > bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java, line 135
bq. > <https://reviews.apache.org/r/3926/diff/1/?file=75426#file75426line135>
bq. >
bq. > if more than two bookie server try to create COOKIE_PATH at the same time, it would fail. it would be better to catch NodeExists Exception.
Fixed
- Ivan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3926/#review5766
-----------------------------------------------------------
On 2012-02-16 16:15:58, Ivan Kelly wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3926/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-02-16 16:15:58)
bq.
bq.
bq. Review request for bookkeeper.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
bq.
bq. 1) A disk is replaced or ledger index is removed by a sloppy admin.
bq. 2) A disk is not mounted when a bookie machine is restarted.
bq.
bq. We need a mechanism to prevent such incorrect responses.
bq.
bq. Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
bq.
bq.
bq. This addresses bug BOOKKEEPER-163.
bq. https://issues.apache.org/jira/browse/BOOKKEEPER-163
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. bookkeeper-server/pom.xml 601104f
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java 2fb7c6c
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieException.java 1a5b313
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java aca66e6
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCache.java 3e96d46
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieJournalTest.java 2df0ed0
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieLayoutVersionTest.java 10f9538
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CookieTest.java PRE-CREATION
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java f661e90
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/UpgradeTest.java PRE-CREATION
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java 99258ac
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java 015e4e4
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BaseTestCase.java dada67a
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java ea51118
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieZKExpireTest.java 5ed7061
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/ZooKeeperUtil.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3926/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Ivan
bq.
bq.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227411#comment-13227411 ]
Sijie Guo commented on BOOKKEEPER-163:
--------------------------------------
yes. I remembered hadoop used 'ln' command to do upgrading too. And it also batched hardlinking to make upgrading fast. I think we can refer its experience.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227390#comment-13227390 ]
Ivan Kelly commented on BOOKKEEPER-163:
---------------------------------------
Hmm, this is true. The problem here is that hardlinking isn't built into java yet (it exists in jdk7, but thats not in general usage). I guess I can do a check for /usr/bin/ln and if it doesn't exist, do a copy.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227452#comment-13227452 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-163:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3926/
-----------------------------------------------------------
(Updated 2012-03-12 12:12:03.757647)
Review request for bookkeeper.
Summary
-------
bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
1) A disk is replaced or ledger index is removed by a sloppy admin.
2) A disk is not mounted when a bookie machine is restarted.
We need a mechanism to prevent such incorrect responses.
Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
This addresses bug BOOKKEEPER-163.
https://issues.apache.org/jira/browse/BOOKKEEPER-163
Diffs (updated)
-----
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCache.java 19295bb
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieJournalTest.java bc7a703
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieLayoutVersionTest.java 10f9538
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java 82f01e8
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java d1427b7
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieException.java 1a5b313
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java PRE-CREATION
bookkeeper-server/pom.xml 601104f
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java 0d66b41
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java 016289d
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CookieTest.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java 281f729
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/UpgradeTest.java eb08479
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java 99258ac
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java 015e4e4
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BaseTestCase.java dada67a
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java ea51118
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieZKExpireTest.java 5ed7061
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/ZooKeeperUtil.java PRE-CREATION
Diff: https://reviews.apache.org/r/3926/diff
Testing
-------
Thanks,
Ivan
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209473#comment-13209473 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-163:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3926/
-----------------------------------------------------------
Review request for bookkeeper.
Summary
-------
bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
1) A disk is replaced or ledger index is removed by a sloppy admin.
2) A disk is not mounted when a bookie machine is restarted.
We need a mechanism to prevent such incorrect responses.
Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
This addresses bug BOOKKEEPER-163.
https://issues.apache.org/jira/browse/BOOKKEEPER-163
Diffs
-----
bookkeeper-server/pom.xml 601104f
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java 2fb7c6c
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieException.java 1a5b313
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java aca66e6
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCache.java 3e96d46
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieJournalTest.java 2df0ed0
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieLayoutVersionTest.java 10f9538
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CookieTest.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java f661e90
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/UpgradeTest.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java 99258ac
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java 015e4e4
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BaseTestCase.java dada67a
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java ea51118
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieZKExpireTest.java 5ed7061
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/ZooKeeperUtil.java PRE-CREATION
Diff: https://reviews.apache.org/r/3926/diff
Testing
-------
Thanks,
Ivan
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Ivan Kelly (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly reassigned BOOKKEEPER-163:
-------------------------------------
Assignee: Ivan Kelly
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227512#comment-13227512 ]
Sijie Guo commented on BOOKKEEPER-163:
--------------------------------------
I am OK for the new patch, +1. Although it seems that hadoop-common jar introduces lot of dependencies, which are not be used actually, it seems that we don't have a better library to support hardlink than hadoop common jar. I would like to hear other one's opinion.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Ivan Kelly (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated BOOKKEEPER-163:
----------------------------------
Attachment: BOOKKEEPER-163.diff
new patch addresses Sijie's comments
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225885#comment-13225885 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-163:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3926/#review5766
-----------------------------------------------------------
most of the patch is good beside some small issues. I commented them inline.
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java
<https://reviews.apache.org/r/3926/#comment12563>
if more than two bookie server try to create COOKIE_PATH at the same time, it would fail. it would be better to catch NodeExists Exception.
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java
<https://reviews.apache.org/r/3926/#comment12565>
it seems that you missed that file 'lastId' and 'lastMark'
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java
<https://reviews.apache.org/r/3926/#comment12564>
it would be better to copy data to a temp directory, such as upgrading, then rename the upgrading directory to current.
otherwise, if the upgrading is broken after created current directory, but user restarts bookie server with this upgrade-failed directory, bookie server would treat it as a new environment.
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CookieTest.java
<https://reviews.apache.org/r/3926/#comment12566>
for a new disk (new directory) added case, it would not cause data missing.
could we provide a tool to upgrade a bookie's cookie when adding a new directory?
- Sijie
On 2012-02-16 16:15:58, Ivan Kelly wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3926/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-02-16 16:15:58)
bq.
bq.
bq. Review request for bookkeeper.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
bq.
bq. 1) A disk is replaced or ledger index is removed by a sloppy admin.
bq. 2) A disk is not mounted when a bookie machine is restarted.
bq.
bq. We need a mechanism to prevent such incorrect responses.
bq.
bq. Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
bq.
bq.
bq. This addresses bug BOOKKEEPER-163.
bq. https://issues.apache.org/jira/browse/BOOKKEEPER-163
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. bookkeeper-server/pom.xml 601104f
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java 2fb7c6c
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieException.java 1a5b313
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java aca66e6
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCache.java 3e96d46
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieJournalTest.java 2df0ed0
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieLayoutVersionTest.java 10f9538
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CookieTest.java PRE-CREATION
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java f661e90
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/UpgradeTest.java PRE-CREATION
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java 99258ac
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java 015e4e4
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BaseTestCase.java dada67a
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java ea51118
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieZKExpireTest.java 5ed7061
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/ZooKeeperUtil.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3926/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Ivan
bq.
bq.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225974#comment-13225974 ]
Ivan Kelly commented on BOOKKEEPER-163:
---------------------------------------
There's a few conflicts with trunk, which im sorting out now. I'll upload a new patch when im done.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209472#comment-13209472 ]
Ivan Kelly commented on BOOKKEEPER-163:
---------------------------------------
Patch applies on top of BOOKKEEPER-172 patch
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Ivan Kelly (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated BOOKKEEPER-163:
----------------------------------
Attachment: BOOKKEEPER-163.diff
Irritatingly, Hardlink doesn't use any 3rd party deps at all. New patch excludes all of hadoop-common's transactive deps. It's ugly, but i think it's the only option we have until jdk7 is commonplace.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Ivan Kelly (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated BOOKKEEPER-163:
----------------------------------
Fix Version/s: 4.1.0
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Fix For: 4.1.0
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227516#comment-13227516 ]
Sijie Guo commented on BOOKKEEPER-163:
--------------------------------------
BTW, hadoop-common jar introduces protobuf-2.4.0, which is higher than 2.3.0.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228228#comment-13228228 ]
Sijie Guo commented on BOOKKEEPER-163:
--------------------------------------
+1 for new patch.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Ivan Kelly (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated BOOKKEEPER-163:
----------------------------------
Attachment: BOOKKEEPER-163.diff
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226022#comment-13226022 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-163:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3926/
-----------------------------------------------------------
(Updated 2012-03-09 11:23:54.722810)
Review request for bookkeeper.
Summary
-------
bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
1) A disk is replaced or ledger index is removed by a sloppy admin.
2) A disk is not mounted when a bookie machine is restarted.
We need a mechanism to prevent such incorrect responses.
Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
This addresses bug BOOKKEEPER-163.
https://issues.apache.org/jira/browse/BOOKKEEPER-163
Diffs (updated)
-----
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieZKExpireTest.java 5ed7061
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/ZooKeeperUtil.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java ea51118
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BaseTestCase.java dada67a
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java 99258ac
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java 015e4e4
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java 281f729
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/UpgradeTest.java eb08479
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CookieTest.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java 016289d
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieLayoutVersionTest.java 10f9538
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCache.java 19295bb
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieJournalTest.java bc7a703
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java 82f01e8
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java d1427b7
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieException.java 1a5b313
bookkeeper-server/pom.xml 601104f
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java 0d66b41
Diff: https://reviews.apache.org/r/3926/diff
Testing
-------
Thanks,
Ivan
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228252#comment-13228252 ]
Hudson commented on BOOKKEEPER-163:
-----------------------------------
Integrated in bookkeeper-trunk #402 (See [https://builds.apache.org/job/bookkeeper-trunk/402/])
BOOKKEEPER-163: Prevent incorrect NoSuchLedgerException for readLastConfirmed. (ivank via sijie) (Revision 1299984)
Result = ABORTED
sijie :
Files :
* /zookeeper/bookkeeper/trunk/CHANGES.txt
* /zookeeper/bookkeeper/trunk/bookkeeper-server/pom.xml
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieException.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCache.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieJournalTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieLayoutVersionTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CookieTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/UpgradeTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BaseTestCase.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieZKExpireTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/ZooKeeperUtil.java
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "Ivan Kelly (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated BOOKKEEPER-163:
----------------------------------
Attachment: BOOKKEEPER-163.diff
New patch uses hardlinking rather than copying.
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect
NoSuchLedgerException for readLastConfirmed.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227563#comment-13227563 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-163:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3926/
-----------------------------------------------------------
(Updated 2012-03-12 14:21:17.503815)
Review request for bookkeeper.
Summary
-------
bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
1) A disk is replaced or ledger index is removed by a sloppy admin.
2) A disk is not mounted when a bookie machine is restarted.
We need a mechanism to prevent such incorrect responses.
Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
This addresses bug BOOKKEEPER-163.
https://issues.apache.org/jira/browse/BOOKKEEPER-163
Diffs (updated)
-----
bookkeeper-server/pom.xml 601104f
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java 0d66b41
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieException.java 1a5b313
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java 82f01e8
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileSystemUpgrade.java d1427b7
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCache.java 19295bb
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieJournalTest.java bc7a703
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieLayoutVersionTest.java 10f9538
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java 016289d
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CookieTest.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java 281f729
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/UpgradeTest.java eb08479
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java 99258ac
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java 015e4e4
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BaseTestCase.java dada67a
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java ea51118
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieZKExpireTest.java 5ed7061
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/ZooKeeperUtil.java PRE-CREATION
Diff: https://reviews.apache.org/r/3926/diff
Testing
-------
Thanks,
Ivan
> Prevent incorrect NoSuchLedgerException for readLastConfirmed.
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-163
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff
>
>
> bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response.
> 1) A disk is replaced or ledger index is removed by a sloppy admin.
> 2) A disk is not mounted when a bookie machine is restarted.
> We need a mechanism to prevent such incorrect responses.
> Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira