You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Sijie Guo (Created) (JIRA)" <ji...@apache.org> on 2012/03/22 09:14:22 UTC
[jira] [Created] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
---------------------------------------------------------------------------------------------------
Key: BOOKKEEPER-191
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
Project: Bookkeeper
Issue Type: Improvement
Reporter: Sijie Guo
Assignee: Sijie Guo
currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430751#comment-13430751 ]
Sijie Guo commented on BOOKKEEPER-191:
--------------------------------------
> When an op is pushed onto the TopicOpQueue, the op is first put on the work queue for that topic, and then run if it is the only thing in the queue. If it is not the only thing in the queue it will wait. Then it will be run when another op(sync or async) finishes and runs popAndRunNext().
Yes. that is how TopicOpQueue works. But for performance consideration, PersistOp takes optimistic mechanism: it assign a message seq id for the message and call BookKeeper#asyncAddEntry to issue an addEntry operation. So the next persist op doesn't need to wait the previous one to callback, BookKeeper could guarantee its order and correctness.
Considering how persist op works, the only chance to issue change ledger op is in the persist's callback when those on-the-fly addEntry operations finished. And the time between planning changing ledger and issuing change-ledger op, other persist op could be put in queue, so we have to queue them until change-ledger op executed.
A detail explanation is described as below:
1) several persist ops are issued. the topicqueue is as below:
(persist 1)
(persist 2)
...
(persist K)
2) persist K executes. At this point, the previous K-1 requests has been issued, but they might still not callback. K detects it is the last entry of the ledger. But it could not issue change ledger op now, other wise the change ledger op would be executed immediately after K send addEntry request. It is dangerous since all K requests might still wait for the callback from BookKeeper.
so the only chance to issue change ledger op is when persist K is callback (BookKeeper guarantees all previous K-1 has succeed when K succeed).
(persist 1) => addEntry
(persist 2) => addEntry
...
(persist K) <= executed now
(change ledger op) (dangerous, if issued immediately by persist K)
3) so the change ledger op is issued when persist K callback. during this time period, other persist ops might still be put in the op queue.
(persist K) => addEntry
...
(persist N)
(persist M)
(change ledger op) (issued by callback of persist K)
so we have to queue those persist requests added between issuing addEntry(K) and addEntry(K) callback.
hope the explanation works for you.
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254508#comment-13254508 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-191:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4731/
-----------------------------------------------------------
Review request for bookkeeper.
Summary
-------
currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
This addresses bug BOOKKEEPER-191.
https://issues.apache.org/jira/browse/BOOKKEEPER-191
Diffs
-----
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 15162b0
hedwig-server/src/main/java/org/apache/hedwig/server/common/ServerConfiguration.java 791c704
hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java 95ee04c
hedwig-server/src/test/java/org/apache/hedwig/server/persistence/TestBookkeeperPersistenceManagerWhiteBox.java 47c7030
Diff: https://reviews.apache.org/r/4731/diff
Testing
-------
Thanks,
Sijie
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-191:
---------------------------------
Attachment: BOOKKEEPER-191.diff
thanks Ivan for reviewing.
update the patch to address Ivan's comments.
review board: https://reviews.apache.org/r/4731/diff/4/
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430962#comment-13430962 ]
Ivan Kelly commented on BOOKKEEPER-191:
---------------------------------------
Ah, you are right. I didn't see that async add was being used there. Really my issue with the fix is that another request queue is being used, when we already have one request queue. And adding this other queue adds complexity, which I think is unnecessary.
I think a simpler solution would be to insert a ChangeLedgerOp into the topic queue, and have this op wait until all previous add ops have completed[1]. Having the ChangeLedgerOp in the queue will prevent any new PersistOp requests from running, and they'll just be queued in the TopicOpQueue as normal.
[1] I'm unsure how to do this bit. Really we need an AtomicInt which PersistOp increments and which runs a callback when it reaches 0, the callback then runs the ChangeLedgerOp logic.
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Sijie Guo (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-191:
---------------------------------
Attachment: BK-191.diff
attach a patch to change ledger by providing size.
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Attachments: BK-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430998#comment-13430998 ]
Ivan Kelly commented on BOOKKEEPER-191:
---------------------------------------
Hmmm, yes, I can't find a clean way to do this. I guess we need to extra queue. I have a few more comments. I'll put them in review board (mostly renames etc).
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430380#comment-13430380 ]
Sijie Guo commented on BOOKKEEPER-191:
--------------------------------------
@Ivan
ChangeLedgerOp is asynchronous op, but PersisteOp is synchronous op. so when the ChangeLedgerOp is executing, the previous persist ops might be still on-the-fly. so doChangeLedger is to avoid close ledger until all previous requests are finished. then it is safe to change ledger.
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-191:
---------------------------------
Attachment: BOOKKEEPER-191.diff
update the patch to latest trunk.
review board: https://reviews.apache.org/r/4731/diff/3/
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430409#comment-13430409 ]
Ivan Kelly commented on BOOKKEEPER-191:
---------------------------------------
{quote}
ChangeLedgerOp is asynchronous op, but PersisteOp is synchronous op. so when the ChangeLedgerOp is executing, the previous persist ops might be still on-the-fly. so doChangeLedger is to avoid close ledger until all previous requests are finished. then it is safe to change ledger.
{quote}
As I understand it, thats not how TopicOpQueuer works. Only one op is allows to run for a topic at a time whether it's sync or async. When an op is pushed onto the TopicOpQueue, the op is first put on the work queue for that topic, and then run if it is the only thing in the queue. If it is not the only thing in the queue it will wait. Then it will be run when another op(sync or async) finishes and runs popAndRunNext().
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431922#comment-13431922 ]
Hudson commented on BOOKKEEPER-191:
-----------------------------------
Integrated in bookkeeper-trunk #641 (See [https://builds.apache.org/job/bookkeeper-trunk/641/])
BOOKKEEPER-191: Hub server should change ledger to write, so consumed messages have chance to be garbage collected. (sijie via ivank) (Revision 1371255)
Result = FAILURE
ivank :
Files :
* /zookeeper/bookkeeper/trunk/CHANGES.txt
* /zookeeper/bookkeeper/trunk/hedwig-server/src/main/java/org/apache/hedwig/server/common/ServerConfiguration.java
* /zookeeper/bookkeeper/trunk/hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java
* /zookeeper/bookkeeper/trunk/hedwig-server/src/test/java/org/apache/hedwig/server/persistence/TestBookkeeperPersistenceManagerWhiteBox.java
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431781#comment-13431781 ]
Ivan Kelly commented on BOOKKEEPER-191:
---------------------------------------
lgtm +1. Committing
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-191:
---------------------------------
Attachment: BOOKKEEPER-191.diff
attach a new patch which rebased to latest trunk.
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-191) Hub server should change ledger
to write, so consumed messages have chance to be garbage collected.
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430983#comment-13430983 ]
Sijie Guo commented on BOOKKEEPER-191:
--------------------------------------
I think the reason why I introduced another pending queue is you couldn't prevent requests being added between the op which inserted ChangeLedgerOp and the ChangeLedgerOp.
{op k} ------------------> issue ChangeLedgerOp
{op l}
...
{op n}
{changeLedgerOp issed by op k}
when op k or the callback of op k said it wants to changeLedger, it inserted a ChangeLedgerOp into the tail of queue. but the ChangeLedgerOp just has knowledge about the operations before op k, but no idea about the ops added between op k and the changeLedgerOp.
this implementation here is quite different with ManagedLedger's implementation. For ManageLedger's implementation, it used a combination of (ledgerid, entryid) as message id, so there is no strong dependency between the operations. but for BookKeeperPersistenceManager, it used an incrementing number as message id, so there is stronger dependency between operations than ManagedLedger. so I think it is quite difficult to use only one AtomicInt to achieve change ledger semantic.
> Hub server should change ledger to write, so consumed messages have chance to be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't change ownership, all entries will be added to this ledger. so those consumed messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira