You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Sijie Guo (Created) (JIRA)" <ji...@apache.org> on 2012/04/01 17:38:28 UTC
[jira] [Created] (BOOKKEEPER-203) improve ledger manager interface
to remove zookeeper dependency on metadata operations.
improve ledger manager interface to remove zookeeper dependency on metadata operations.
---------------------------------------------------------------------------------------
Key: BOOKKEEPER-203
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-client, bookkeeper-server
Reporter: Sijie Guo
Assignee: Sijie Guo
we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13395943#comment-13395943 ]
Ivan Kelly commented on BOOKKEEPER-203:
---------------------------------------
BOOKKEEPER-203.diff_v6 doesn't apply cleanly to trunk? Did you generate the patch against the correct branch? It looks like the clash is against BOOKKEEPER-183.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5, BOOKKEEPER-203.diff_v6
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294328#comment-13294328 ]
Ivan Kelly commented on BOOKKEEPER-203:
---------------------------------------
Patch looks good. There's some small issues with it.
- Needs to be rebased on trunk
- MetaStoreException is not used. Is in wrong patch i guess?
- BookKeeper.java patch removed the throwing of NoSuchLedgerExistsException, which we added in BOOKKEEPER-273. Is this intentional?
- there are two overloads of getClass where only one is ever used. setClass is unused.
- LedgerManagerFactory#newLedgerManagerFactory breaks getLedgerManagerType(). if factoryClass is null and layout is null, it should check getLedgerManagerType before setting factoryClass to FlatLedgerManagerType or HierarchicalLedgerManagerType.
- This does introduce some incompatiblity. A 4.1.0 client will not work with an installation that has been initialised with the LedgerLayout code from this patch. This should be fine though, as LedgerLayout is only ever written on cluster initialization. However, I think we should change this to use protobufs so that any other changes like this are more BC friendly. Not in this patch though.
- You may as well you MultiLedgerManagerBaseTestCase for the places where you use MultiLedgerManagerTestCase and get rid of MultiLedgerManagerTestCase.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396685#comment-13396685 ]
Hudson commented on BOOKKEEPER-203:
-----------------------------------
Integrated in bookkeeper-trunk #567 (See [https://builds.apache.org/job/bookkeeper-trunk/567/])
BOOKKEEPER-203: improve ledger manager interface to remove zookeeper dependency on metadata operations. (sijie via ivank) (Revision 1351646)
Result = ABORTED
ivank :
Files :
* /zookeeper/bookkeeper/trunk/CHANGES.txt
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerCreateOp.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerDeleteOp.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerOpenOp.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ActiveLedgerManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManagerFactory.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManagerFactory.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerLayout.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManagerFactory.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ZkVersion.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/util/ReflectionUtils.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/versioning
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/versioning/Version.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/versioning/Versioned.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/LedgerCacheTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalAsyncLedgerOpsTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieFailureTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieReadWriteTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieRecoveryTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalLedgerDeleteTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerLayoutTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/TestLedgerManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/TestZkVersion.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/AsyncLedgerOpsTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookKeeperClusterTestCase.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieReadWriteTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/LedgerDeleteTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerMultiDigestTestCase.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerTestCase.java
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5, BOOKKEEPER-203.diff_v6, BOOKKEEPER-203.diff_v7
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396337#comment-13396337 ]
Sijie Guo commented on BOOKKEEPER-203:
--------------------------------------
@Ivan,
Versioned is added as a generic interface. I was planning to make LedgerMetadata as Versioned interface, but seems that there are too much modifications. so I don't do the change.
But the idea I kept Versioned here is for completeness of versioning interface, and it would be used in Hedwig Meta Manager to write topic persistence info and owner info. (BOOKKEEPER-250 and BOOKKEEPER-259)
meta package is used to put metadata related things of BookKeeper, while Version and Versioned are more generic abstraction than BookKeeper meta, it would be used in both BookKeeper and Hedwig metadata manager. So I put them in a separated versioning package.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5, BOOKKEEPER-203.diff_v6, BOOKKEEPER-203.diff_v7
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243756#comment-13243756 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-203:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4603/
-----------------------------------------------------------
Review request for bookkeeper.
Summary
-------
we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
This addresses bug BOOKKEEPER-203.
https://issues.apache.org/jira/browse/BOOKKEEPER-203
Diffs
-----
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java ee38862
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java 2eed192
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java 17c70f1
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java 911c660
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java 490c130
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java b9cd624
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerCreateOp.java f0a165f
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerDeleteOp.java 5b10a5b
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java e576792
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java 2dcf0e4
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerOpenOp.java 56186ab
bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java e1bd1ff
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java 8fd7df8
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ActiveLedgerManager.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java 1300974
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManagerFactory.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java b573181
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManagerFactory.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerLayout.java 3a3a81e
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManager.java 7d9eb96
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManagerFactory.java b62ad3a
bookkeeper-server/src/main/java/org/apache/bookkeeper/util/ReflectionUtils.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/LedgerCacheTest.java 52cf514
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java 6135250
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java c5350ad
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java 12c1b5a
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalAsyncLedgerOpsTest.java 24b3c12
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieFailureTest.java 70a4ea9
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieReadWriteTest.java 7d97fd4
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieRecoveryTest.java d99156e
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalLedgerDeleteTest.java dbcd65a
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerLayoutTest.java 4784854
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java b8a541b
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/TestLedgerManager.java e07c756
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/AsyncLedgerOpsTest.java d7f153a
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookKeeperClusterTestCase.java c172224
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java e647d68
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieReadWriteTest.java 3a3c8af
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/LedgerDeleteTest.java 2d3593e
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerBaseTestCase.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerTestCase.java PRE-CREATION
Diff: https://reviews.apache.org/r/4603/diff
Testing
-------
Thanks,
Sijie
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-203.diff
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274138#comment-13274138 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-203:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4603/
-----------------------------------------------------------
(Updated 2012-05-13 02:44:58.032203)
Review request for bookkeeper.
Changes
-------
update a new patch, which brought the code to latest trunk and addressed Flavio's suggestions.
Summary
-------
we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
This addresses bug BOOKKEEPER-203.
https://issues.apache.org/jira/browse/BOOKKEEPER-203
Diffs (updated)
-----
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java aa4dec2
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java 4dc71e5
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java c8b5001
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java 4df689e
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java 484a055
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java eb0955b
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java 712bbea
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerCreateOp.java 8c5bce3
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerDeleteOp.java 5b10a5b
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java 6ce576d
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java c699eae
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerOpenOp.java 101ebf3
bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java e1bd1ff
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java ce53461
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ActiveLedgerManager.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java 1300974
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManagerFactory.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java b573181
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManagerFactory.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerLayout.java cab02fe
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManager.java 7d9eb96
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManagerFactory.java 491a3e4
bookkeeper-server/src/main/java/org/apache/bookkeeper/util/ReflectionUtils.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/LedgerCacheTest.java 52cf514
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java 8bd3f6b
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java 1c109a2
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java 12c1b5a
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalAsyncLedgerOpsTest.java 24b3c12
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieFailureTest.java 70a4ea9
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieReadWriteTest.java 7d97fd4
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieRecoveryTest.java d99156e
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalLedgerDeleteTest.java dbcd65a
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerLayoutTest.java 4784854
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java b8a541b
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/TestLedgerManager.java e07c756
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/AsyncLedgerOpsTest.java d7f153a
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookKeeperClusterTestCase.java a81067d
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java e647d68
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieReadWriteTest.java 3a3c8af
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/LedgerDeleteTest.java 2d3593e
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerBaseTestCase.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerTestCase.java PRE-CREATION
Diff: https://reviews.apache.org/r/4603/diff
Testing
-------
Thanks,
Sijie
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274113#comment-13274113 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-203:
----------------------------------------------------------
bq. On 2012-04-03 21:17:04, fpj wrote:
bq. > It looks pretty good for me, I just have a few small comments.
Thanks Flavio for reviewing it. I would like to bring the patch to latest trunk and update it with your comments.
bq. On 2012-04-03 21:17:04, fpj wrote:
bq. > bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java, line 178
bq. > <https://reviews.apache.org/r/4603/diff/1/?file=97866#file97866line178>
bq. >
bq. > It doesn't look like we need to make this call public. If it is for testing, we could simply use inheritance, no?
we use parseConfig in ledgermanager not for testing, which is another package.
bq. On 2012-04-03 21:17:04, fpj wrote:
bq. > bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java, line 146
bq. > <https://reviews.apache.org/r/4603/diff/1/?file=97868#file97868line146>
bq. >
bq. > I'm wondering if it is ok to have a reference specific to zk here. Is the idea here that for every new backend we add, we include required system properties here?
zkLedgersRootPath not only for zookeeper-based ledger manager to store ledger metadata. under zkLedgersRootPath there is a 'layout' znode, which describes the metadata layout and which ledger manager it used for current bookkeeper cluster.
bq. On 2012-04-03 21:17:04, fpj wrote:
bq. > bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ActiveLedgerManager.java, line 30
bq. > <https://reviews.apache.org/r/4603/diff/1/?file=97870#file97870line30>
bq. >
bq. > I wonder if ActiveLedgerManager is a good name for this class. It doesn't indicate that it is related to the bookie side, and it might be good to refer to it. But, I don't feel too strongly about it, since the comment before the class says that it is for bookies.
I don't have a good name for it also, just used its original name before we abstracted ledger manager interface.
- Sijie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4603/#review6616
-----------------------------------------------------------
On 2012-04-01 15:56:33, Sijie Guo wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4603/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-01 15:56:33)
bq.
bq.
bq. Review request for bookkeeper.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
bq.
bq.
bq. This addresses bug BOOKKEEPER-203.
bq. https://issues.apache.org/jira/browse/BOOKKEEPER-203
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java ee38862
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java 2eed192
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java 17c70f1
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java 911c660
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java 490c130
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java b9cd624
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerCreateOp.java f0a165f
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerDeleteOp.java 5b10a5b
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java e576792
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java 2dcf0e4
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerOpenOp.java 56186ab
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java e1bd1ff
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java 8fd7df8
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ActiveLedgerManager.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java 1300974
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManagerFactory.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java b573181
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManagerFactory.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerLayout.java 3a3a81e
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManager.java 7d9eb96
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManagerFactory.java b62ad3a
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/util/ReflectionUtils.java PRE-CREATION
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/LedgerCacheTest.java 52cf514
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java 6135250
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java c5350ad
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java 12c1b5a
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalAsyncLedgerOpsTest.java 24b3c12
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieFailureTest.java 70a4ea9
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieReadWriteTest.java 7d97fd4
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieRecoveryTest.java d99156e
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalLedgerDeleteTest.java dbcd65a
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerLayoutTest.java 4784854
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java b8a541b
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/TestLedgerManager.java e07c756
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/AsyncLedgerOpsTest.java d7f153a
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookKeeperClusterTestCase.java c172224
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java e647d68
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieReadWriteTest.java 3a3c8af
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/LedgerDeleteTest.java 2d3593e
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerBaseTestCase.java PRE-CREATION
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerTestCase.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/4603/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Sijie
bq.
bq.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13395814#comment-13395814 ]
Ivan Kelly commented on BOOKKEEPER-203:
---------------------------------------
{quote}If everything is run as MultiLedgerManagerBaseTestCase, it would take a long time to run. {quote}
This is true. Would it be a big change to rename it to MultiLedgerManagerMultiDigestTestCase, to make things clearer?
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294330#comment-13294330 ]
Ivan Kelly commented on BOOKKEEPER-203:
---------------------------------------
ah, ignore first comment.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396111#comment-13396111 ]
Ivan Kelly commented on BOOKKEEPER-203:
---------------------------------------
One last thing (unfortunately it wasn't in the patch i reviewed last week). I think Version and Versioned should be in the meta package and not in a package of their own. Actually, Versioned doesn't even seem to be used at all. I guess it's there for another patch you have planned?
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5, BOOKKEEPER-203.diff_v6, BOOKKEEPER-203.diff_v7
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Flavio Junqueira (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245764#comment-13245764 ]
Flavio Junqueira commented on BOOKKEEPER-203:
---------------------------------------------
Even if we +1 this jira, I was wondering if we should move it to 4.2.0. This jira is a sub-task of BOOKKEPER-181, and we have marked it for 4.2.0.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-203.diff
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13395766#comment-13395766 ]
Sijie Guo commented on BOOKKEEPER-203:
--------------------------------------
@Ivan,
{code}
HierarchicalAsyncLedgerOpsTest -> AsyncLedgerOpsTest (BaseTestCase)
HierarchicalBookieReadWriteTest -> BookieReadWriteTest (BaseTestCase)
HierarchicalBookieFailureTest -> BookieFailureTest (BaseTestCase)
HierarchicalBookieRecoveryTest -> BookieRecoveryTest (BaseTestCase)
HierarchicalLedgerDeleteTest -> LedgerDeleteTest (BookKeeperClusterTestCase)
{code}
MultiLedgerManagerBaseTestCase & MultiLedgerManagerTestCase are used to remove HierarchicalxxxTests to let them run as parameterized tests.
{quote}
So MultiLedgerManagerBaseTestCase is really a superset of MultiLedgerManagerTestCase. I don't see any harm in just using MultiLedgerManagerBaseTestCase for all.
{quote}
If everything is run as MultiLedgerManagerBaseTestCase, it would take a long time to run. For example, LedgerDeleteTest just tests ledger deletion functionality. so we don't need to parameterize the test to run with different digest type. But we have to run with different ledger managers because deletion is related to gc implementation in different ledger managers.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-203) improve ledger manager interface
to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-203:
---------------------------------
Attachment: BOOKKEEPER-203.diff_v3
attach a new patch rebased to latest trunk.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-203) improve ledger manager interface
to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-203:
---------------------------------
Fix Version/s: (was: 4.1.0)
4.2.0
Agreed to move it to 4.2.0.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-203) improve ledger manager interface
to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-203:
---------------------------------
Attachment: BOOKKEEPER-203.diff
create a patch to improve ledger manager interface
1) split ledger manager interface into client/server part, client side is LedgerManager, server side is ActiveLedgerManager.
2) remove zookeeper dependency on metadata operations, provides a well-defined metadata operation interface.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Attachments: BOOKKEEPER-203.diff
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-203) improve ledger manager interface
to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-203:
---------------------------------
Attachment: BOOKKEEPER-203.diff_v2
attach a new patch, which brought the code to latest trunk and addressed Flavio's suggestions.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396586#comment-13396586 ]
Ivan Kelly commented on BOOKKEEPER-203:
---------------------------------------
Ok, I'll take one last pass over the patch, but i think it's ready to go.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5, BOOKKEEPER-203.diff_v6, BOOKKEEPER-203.diff_v7
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-203) improve ledger manager interface
to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-203:
---------------------------------
Attachment: BOOKKEEPER-203.diff_v4
new patch 1) updated to latest trunk 2) introduce Version interface to remove zookeeper integer version dependency. because not every storage support integer version.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396664#comment-13396664 ]
Ivan Kelly commented on BOOKKEEPER-203:
---------------------------------------
looks good +1
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5, BOOKKEEPER-203.diff_v6, BOOKKEEPER-203.diff_v7
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393673#comment-13393673 ]
Sijie Guo commented on BOOKKEEPER-203:
--------------------------------------
{quote}
You may as well you MultiLedgerManagerBaseTestCase for the places where you use MultiLedgerManagerTestCase and get rid of MultiLedgerManagerTestCase.
{quote}
MultiLedgerManagerBaseTestCase is used to test different digest type and ledger manager, which gets rid of adding Hierarchical prefixed test cases.
MultiLedgerManagerTestCase is used to test different ledger manager.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245762#comment-13245762 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-203:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4603/#review6616
-----------------------------------------------------------
It looks pretty good for me, I just have a few small comments.
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java
<https://reviews.apache.org/r/4603/#comment14294>
For clarity, can we rename it to MetaStoreException or MetadataStoreException?
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java
<https://reviews.apache.org/r/4603/#comment14397>
MetaStoreException
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
<https://reviews.apache.org/r/4603/#comment14401>
It doesn't look like we need to make this call public. If it is for testing, we could simply use inheritance, no?
bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java
<https://reviews.apache.org/r/4603/#comment14404>
I'm wondering if it is ok to have a reference specific to zk here. Is the idea here that for every new backend we add, we include required system properties here?
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ActiveLedgerManager.java
<https://reviews.apache.org/r/4603/#comment14417>
You may want to say bookie instead of server side.
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ActiveLedgerManager.java
<https://reviews.apache.org/r/4603/#comment14416>
I wonder if ActiveLedgerManager is a good name for this class. It doesn't indicate that it is related to the bookie side, and it might be good to refer to it. But, I don't feel too strongly about it, since the comment before the class says that it is for bookies.
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManager.java
<https://reviews.apache.org/r/4603/#comment14405>
The comment here should not refer to zk, right?
- fpj
On 2012-04-01 15:56:33, Sijie Guo wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4603/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-01 15:56:33)
bq.
bq.
bq. Review request for bookkeeper.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
bq.
bq.
bq. This addresses bug BOOKKEEPER-203.
bq. https://issues.apache.org/jira/browse/BOOKKEEPER-203
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java ee38862
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java 2eed192
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java 17c70f1
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java 911c660
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java 490c130
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java b9cd624
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerCreateOp.java f0a165f
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerDeleteOp.java 5b10a5b
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java e576792
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java 2dcf0e4
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerOpenOp.java 56186ab
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java e1bd1ff
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java 8fd7df8
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ActiveLedgerManager.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java 1300974
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManagerFactory.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java b573181
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManagerFactory.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerLayout.java 3a3a81e
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManager.java 7d9eb96
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManagerFactory.java b62ad3a
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/util/ReflectionUtils.java PRE-CREATION
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/LedgerCacheTest.java 52cf514
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java 6135250
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestFencing.java c5350ad
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java 12c1b5a
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalAsyncLedgerOpsTest.java 24b3c12
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieFailureTest.java 70a4ea9
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieReadWriteTest.java 7d97fd4
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalBookieRecoveryTest.java d99156e
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalLedgerDeleteTest.java dbcd65a
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerLayoutTest.java 4784854
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java b8a541b
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/TestLedgerManager.java e07c756
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/AsyncLedgerOpsTest.java d7f153a
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookKeeperClusterTestCase.java c172224
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieFailureTest.java e647d68
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieReadWriteTest.java 3a3c8af
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/LedgerDeleteTest.java 2d3593e
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerBaseTestCase.java PRE-CREATION
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerTestCase.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/4603/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Sijie
bq.
bq.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-203.diff
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13395481#comment-13395481 ]
Ivan Kelly commented on BOOKKEEPER-203:
---------------------------------------
{code}
MultiLedgerManagerBaseTestCase is used to test different digest type and ledger manager, which gets rid of adding Hierarchical prefixed test cases.
MultiLedgerManagerTestCase is used to test different ledger manager.
{code}
So MultiLedgerManagerBaseTestCase is really a superset of MultiLedgerManagerTestCase. I don't see any harm in just using MultiLedgerManagerBaseTestCase for all. The reason I wrote some test cases which didn't use the BaseTestCase, was because with Parameterized junit tests, it's not possible to run individual tests (i.e. mvn test -Dtest=TestFencing#testBasicFencing), which I found annoying. However, if the test is going to use a Parameterized junit test in any case, we should just run everything.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-203) improve ledger manager interface
to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-203:
---------------------------------
Attachment: BOOKKEEPER-203.diff_v5
update new patch addressing Ivan's comments.
1) rebased to latest trunk
2) remove unused MetaStoreException and setClass in ReflectionUtils
3) improve LedgerManagerFactory#newLedgerManagerFactory for backward compatibility.
4) add uninitialize in LedgerManagerFactory to release resources used by ledger manager factory.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-203) improve ledger manager interface
to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-203:
---------------------------------
Attachment: BOOKKEEPER-203.diff_v7
Thanks Ivan for reminding.
Attach a new patch rebased on latest trunk.
review board: https://reviews.apache.org/r/4603/diff/6
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5, BOOKKEEPER-203.diff_v6, BOOKKEEPER-203.diff_v7
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-203) improve ledger manager
interface to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294363#comment-13294363 ]
Sijie Guo commented on BOOKKEEPER-203:
--------------------------------------
Ivan, thanks for reviewing.
{quote}
MetaStoreException is not used. Is in wrong patch i guess?
{quote}
I think it is introduced when I am working on BOOKKEEPER-204. would remove it in next patch.
{quote}
BookKeeper.java patch removed the throwing of NoSuchLedgerExistsException, which we added in BOOKKEEPER-273. Is this intentional?
{quote}
Actually, I put in AbstractZkLedgerManager, because NONODE is zookeeper's return code. You could refer it as below:
{code}
+ @Override
+ public void deleteLedger(final long ledgerId, final GenericCallback<Void> cb) {
+ zk.delete(getLedgerPath(ledgerId), -1, new VoidCallback() {
+ @Override
+ public void processResult(int rc, String path, Object ctx) {
+ int bkRc;
+ if (rc == KeeperException.Code.NONODE.intValue()) {
+ LOG.warn("Ledger node does not exist in ZooKeeper: ledgerId={}", ledgerId);
+ bkRc = BKException.Code.NoSuchLedgerExistsException;
+ } else if (rc == KeeperException.Code.OK.intValue()) {
+ bkRc = BKException.Code.OK;
+ } else {
+ bkRc = BKException.Code.ZKException;
+ }
+ cb.operationComplete(bkRc, (Void)null);
+ }
+ }, null);
+ }
{code}
{quote}
there are two overloads of getClass where only one is ever used. setClass is unused.
{quote}
ah, ur right. would remove unused methods.
{quote}
LedgerManagerFactory#newLedgerManagerFactory breaks getLedgerManagerType(). if factoryClass is null and layout is null, it should check getLedgerManagerType before setting factoryClass to FlatLedgerManagerType or HierarchicalLedgerManagerType.
{quote}
yes. ur right. would improve it.
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-203) improve ledger manager interface
to remove zookeeper dependency on metadata operations.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-203:
---------------------------------
Attachment: BOOKKEEPER-203.diff_v6
attach a new patch renaming MultiLedgerManagerBaseTestCase to MultiLedgerManagerMultiDigestTestCase
review board: https://reviews.apache.org/r/4603/diff/5/#index_header
> improve ledger manager interface to remove zookeeper dependency on metadata operations.
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-203
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-203
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-client, bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-203.diff, BOOKKEEPER-203.diff_v2, BOOKKEEPER-203.diff_v3, BOOKKEEPER-203.diff_v4, BOOKKEEPER-203.diff_v5, BOOKKEEPER-203.diff_v6
>
>
> we need to improve ledger manager interface to remove zookeeper dependency on metadata operations, so it is easy for us to implement a MetaStore based ledger manager.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira