You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by Jiannan Wang <ji...@yahoo-inc.com> on 2012/10/30 11:17:44 UTC
Review Request: [BOOKKEEPER-442] Failed to deliver messages due to
inconsistency between SubscriptionState and LedgerRanges.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7775/
-----------------------------------------------------------
Review request for bookkeeper.
Description
-------
Since InMemorySubscriptionState and LedgerRanges is maintained separately, there may exist such inconsistent state:
(1). Topic ledgers: L1 [1 ~ 2], L2 [3 ~ ]
(2). Subscriber consumes to 2 and InMemorySubscriptionState is updated successfully but failed when updating subscription state metadata
(3). AbstractSubscriptionManager#MessagesConsumedTask use InMemorySubscriptionState to do garbage collection and L1 is delete
(4). If Hub restarts at this time, old subscription state is read and Hub will try to deliver message from 1
This addresses bug BOOKKEEPER-442.
https://issues.apache.org/jira/browse/BOOKKEEPER-442
Diffs
-----
hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java 40811de
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java bbe8edf
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionState.java 34b54b8
hedwig-server/src/test/java/org/apache/hedwig/server/persistence/TestBookKeeperPersistenceManager.java c645b11
Diff: https://reviews.apache.org/r/7775/diff/
Testing
-------
Two test are added
Thanks,
Jiannan Wang
Re: Review Request: [BOOKKEEPER-442] Failed to deliver messages due to
inconsistency between SubscriptionState and LedgerRanges.
Posted by Jiannan Wang <ji...@yahoo-inc.com>.
> On Nov. 2, 2012, 11:48 a.m., Ivan Kelly wrote:
> > hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java, line 145
> > <https://reviews.apache.org/r/7775/diff/1/?file=182289#file182289line145>
> >
> > Am I correct in thinking that the original logic here was incorrect?
Yeah, I think both of the orignal comment and its implementation code were incorrect.
> On Nov. 2, 2012, 11:48 a.m., Ivan Kelly wrote:
> > hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java, line 237
> > <https://reviews.apache.org/r/7775/diff/1/?file=182288#file182288line237>
> >
> > Are you sure you want to throw a RuntimeException here, as this will probably take the whole hub down.
I don't quit sure about the criterial when to throw a RuntimeException in a system, but it seems use an error logging is Ok here.
- Jiannan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7775/#review13035
-----------------------------------------------------------
On Oct. 30, 2012, 10:17 a.m., Jiannan Wang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7775/
> -----------------------------------------------------------
>
> (Updated Oct. 30, 2012, 10:17 a.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> Since InMemorySubscriptionState and LedgerRanges is maintained separately, there may exist such inconsistent state:
> (1). Topic ledgers: L1 [1 ~ 2], L2 [3 ~ ]
> (2). Subscriber consumes to 2 and InMemorySubscriptionState is updated successfully but failed when updating subscription state metadata
> (3). AbstractSubscriptionManager#MessagesConsumedTask use InMemorySubscriptionState to do garbage collection and L1 is delete
> (4). If Hub restarts at this time, old subscription state is read and Hub will try to deliver message from 1
>
>
> This addresses bug BOOKKEEPER-442.
> https://issues.apache.org/jira/browse/BOOKKEEPER-442
>
>
> Diffs
> -----
>
> hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java 40811de
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java bbe8edf
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionState.java 34b54b8
> hedwig-server/src/test/java/org/apache/hedwig/server/persistence/TestBookKeeperPersistenceManager.java c645b11
>
> Diff: https://reviews.apache.org/r/7775/diff/
>
>
> Testing
> -------
>
> Two test are added
>
>
> Thanks,
>
> Jiannan Wang
>
>
Re: Review Request: [BOOKKEEPER-442] Failed to deliver messages due to
inconsistency between SubscriptionState and LedgerRanges.
Posted by Ivan Kelly <iv...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7775/#review13035
-----------------------------------------------------------
hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java
<https://reviews.apache.org/r/7775/#comment28050>
Are you sure you want to throw a RuntimeException here, as this will probably take the whole hub down.
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java
<https://reviews.apache.org/r/7775/#comment28049>
Am I correct in thinking that the original logic here was incorrect?
- Ivan Kelly
On Oct. 30, 2012, 10:17 a.m., Jiannan Wang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7775/
> -----------------------------------------------------------
>
> (Updated Oct. 30, 2012, 10:17 a.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> Since InMemorySubscriptionState and LedgerRanges is maintained separately, there may exist such inconsistent state:
> (1). Topic ledgers: L1 [1 ~ 2], L2 [3 ~ ]
> (2). Subscriber consumes to 2 and InMemorySubscriptionState is updated successfully but failed when updating subscription state metadata
> (3). AbstractSubscriptionManager#MessagesConsumedTask use InMemorySubscriptionState to do garbage collection and L1 is delete
> (4). If Hub restarts at this time, old subscription state is read and Hub will try to deliver message from 1
>
>
> This addresses bug BOOKKEEPER-442.
> https://issues.apache.org/jira/browse/BOOKKEEPER-442
>
>
> Diffs
> -----
>
> hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java 40811de
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java bbe8edf
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionState.java 34b54b8
> hedwig-server/src/test/java/org/apache/hedwig/server/persistence/TestBookKeeperPersistenceManager.java c645b11
>
> Diff: https://reviews.apache.org/r/7775/diff/
>
>
> Testing
> -------
>
> Two test are added
>
>
> Thanks,
>
> Jiannan Wang
>
>
Re: Review Request: [BOOKKEEPER-442] Failed to deliver messages due to
inconsistency between SubscriptionState and LedgerRanges.
Posted by Ivan Kelly <iv...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7775/#review13724
-----------------------------------------------------------
hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java
<https://reviews.apache.org/r/7775/#comment29409>
Simply logging isn't fully correct either. The client will hang. You need to complete the callback as well.
request.callback.scanFailed(ctx, new PubSubException.UncertainStateException(bke));
It may also make sense to release the topic, to reset the state.
- Ivan Kelly
On Nov. 5, 2012, 10:36 a.m., Jiannan Wang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7775/
> -----------------------------------------------------------
>
> (Updated Nov. 5, 2012, 10:36 a.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> Since InMemorySubscriptionState and LedgerRanges is maintained separately, there may exist such inconsistent state:
> (1). Topic ledgers: L1 [1 ~ 2], L2 [3 ~ ]
> (2). Subscriber consumes to 2 and InMemorySubscriptionState is updated successfully but failed when updating subscription state metadata
> (3). AbstractSubscriptionManager#MessagesConsumedTask use InMemorySubscriptionState to do garbage collection and L1 is delete
> (4). If Hub restarts at this time, old subscription state is read and Hub will try to deliver message from 1
>
>
> This addresses bug BOOKKEEPER-442.
> https://issues.apache.org/jira/browse/BOOKKEEPER-442
>
>
> Diffs
> -----
>
> hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java 40811de
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java bbe8edf
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionState.java 34b54b8
> hedwig-server/src/test/java/org/apache/hedwig/server/persistence/TestBookKeeperPersistenceManager.java c645b11
>
> Diff: https://reviews.apache.org/r/7775/diff/
>
>
> Testing
> -------
>
> Two test are added
>
>
> Thanks,
>
> Jiannan Wang
>
>
Re: Review Request: [BOOKKEEPER-442] Failed to deliver messages due to
inconsistency between SubscriptionState and LedgerRanges.
Posted by Jiannan Wang <ji...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7775/
-----------------------------------------------------------
(Updated Dec. 3, 2012, 12:24 p.m.)
Review request for bookkeeper.
Description
-------
Since InMemorySubscriptionState and LedgerRanges is maintained separately, there may exist such inconsistent state:
(1). Topic ledgers: L1 [1 ~ 2], L2 [3 ~ ]
(2). Subscriber consumes to 2 and InMemorySubscriptionState is updated successfully but failed when updating subscription state metadata
(3). AbstractSubscriptionManager#MessagesConsumedTask use InMemorySubscriptionState to do garbage collection and L1 is delete
(4). If Hub restarts at this time, old subscription state is read and Hub will try to deliver message from 1
This addresses bug BOOKKEEPER-442.
https://issues.apache.org/jira/browse/BOOKKEEPER-442
Diffs (updated)
-----
hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java 40811de
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 31bf505
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionState.java 08994f0
hedwig-server/src/test/java/org/apache/hedwig/server/persistence/TestBookKeeperPersistenceManager.java d582fb6
Diff: https://reviews.apache.org/r/7775/diff/
Testing
-------
Two test are added
Thanks,
Jiannan Wang
Re: Review Request: [BOOKKEEPER-442] Failed to deliver messages due to
inconsistency between SubscriptionState and LedgerRanges.
Posted by Jiannan Wang <ji...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7775/
-----------------------------------------------------------
(Updated Nov. 5, 2012, 10:36 a.m.)
Review request for bookkeeper.
Description
-------
Since InMemorySubscriptionState and LedgerRanges is maintained separately, there may exist such inconsistent state:
(1). Topic ledgers: L1 [1 ~ 2], L2 [3 ~ ]
(2). Subscriber consumes to 2 and InMemorySubscriptionState is updated successfully but failed when updating subscription state metadata
(3). AbstractSubscriptionManager#MessagesConsumedTask use InMemorySubscriptionState to do garbage collection and L1 is delete
(4). If Hub restarts at this time, old subscription state is read and Hub will try to deliver message from 1
This addresses bug BOOKKEEPER-442.
https://issues.apache.org/jira/browse/BOOKKEEPER-442
Diffs (updated)
-----
hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java 40811de
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java bbe8edf
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionState.java 34b54b8
hedwig-server/src/test/java/org/apache/hedwig/server/persistence/TestBookKeeperPersistenceManager.java c645b11
Diff: https://reviews.apache.org/r/7775/diff/
Testing
-------
Two test are added
Thanks,
Jiannan Wang