You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by Sijie Guo <gu...@gmail.com> on 2012/05/10 10:24:54 UTC
Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to kill
other subscription channel when hedwig client is used as a proxy-style
server.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/
-----------------------------------------------------------
Review request for bookkeeper.
Summary
-------
In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
client -> proxy server 1 -> hedwig
\> proxy server 2 />
when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
This addresses bug BOOKKEEPER-252.
https://issues.apache.org/jira/browse/BOOKKEEPER-252
Diffs
-----
hedwig-client/src/main/cpp/inc/hedwig/callback.h 5200d5b
hedwig-client/src/main/cpp/inc/hedwig/subscribe.h fc8da58
hedwig-client/src/main/cpp/lib/channel.cpp ada1e58
hedwig-client/src/main/cpp/lib/clientimpl.h 40e063c
hedwig-client/src/main/cpp/lib/clientimpl.cpp 712de01
hedwig-client/src/main/cpp/lib/data.h d9d2ffd
hedwig-client/src/main/cpp/lib/data.cpp 42fa2ad
hedwig-client/src/main/cpp/lib/publisherimpl.cpp a421e98
hedwig-client/src/main/cpp/lib/subscriberimpl.h 508a495
hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6f7d02e
hedwig-client/src/main/cpp/test/subscribetest.cpp f90502c
hedwig-client/src/main/cpp/test/util.h 780a466
hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 972590b
hedwig-client/src/main/java/org/apache/hedwig/client/data/PubSubData.java 8460de0
hedwig-client/src/main/java/org/apache/hedwig/client/data/TopicSubscriber.java 064cec1
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigPublisher.java 603766c
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 0c8634c
hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java c4affaf
hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionChannelListener.java PRE-CREATION
hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 92f409d
hedwig-protocol/src/main/protobuf/PubSubProtocol.proto b845c40
hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigCommands.java a17f56b
hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 5da8e16
hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 76e0c14
hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 52089f5
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java ba841af
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java 0fdfad0
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/ZkSubscriptionManager.java 4d39335
hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 08994de
hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 5f6e363
hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java d24a86d
hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java 255880b
hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestZkSubscriptionManager.java f6d533f
Diff: https://reviews.apache.org/r/5086/diff
Testing
-------
Thanks,
Sijie
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by Sijie Guo <gu...@gmail.com>.
> On Sept. 14, 2012, 6:18 a.m., Aniruddha Laud wrote:
> > hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java, line 99
> > <https://reviews.apache.org/r/5086/diff/3/?file=154671#file154671line99>
> >
> > Wrap inside a synchronizedSet() ?
>
> Sijie Guo wrote:
> from 'http://docs.oracle.com/javase/6/docs/api/java/util/Collections.html#newSetFromMap(java.util.Map)', it indicates the returned set displays the same ordering, concurrency, and performance characteristics as the backing map.
>
> since I used ConcurrentHashMap, there is no need to synchronizedSet.
chatting with Ivan, CopyOnWriteArraySet is fast on traversal via iterators. Adding/Removing listeners would be rare operations. Using CopyOnWriteArraySet would be a better solution.
- Sijie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/#review11516
-----------------------------------------------------------
On Sept. 14, 2012, 12:07 a.m., Sijie Guo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5086/
> -----------------------------------------------------------
>
> (Updated Sept. 14, 2012, 12:07 a.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
>
> client -> proxy server 1 -> hedwig
> \> proxy server 2 />
>
> when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
>
> we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
>
> the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
>
> so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
>
> when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
>
> in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
>
>
> This addresses bug BOOKKEEPER-252.
> https://issues.apache.org/jira/browse/BOOKKEEPER-252
>
>
> Diffs
> -----
>
> hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
> hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
> hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
> hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
> hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
> hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
> hedwig-client/src/main/cpp/test/util.h 45a6db3
> hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java e3d46ca
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5bcf786
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java 075705a
> hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java PRE-CREATION
> hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
> hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
> hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 7d8e22b
> hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java b9125ca
> hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
> hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
> hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
> hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
> hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
>
> Diff: https://reviews.apache.org/r/5086/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sijie Guo
>
>
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by Sijie Guo <gu...@gmail.com>.
> On Sept. 14, 2012, 6:18 a.m., Aniruddha Laud wrote:
> > hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java, line 99
> > <https://reviews.apache.org/r/5086/diff/3/?file=154671#file154671line99>
> >
> > Wrap inside a synchronizedSet() ?
from 'http://docs.oracle.com/javase/6/docs/api/java/util/Collections.html#newSetFromMap(java.util.Map)', it indicates the returned set displays the same ordering, concurrency, and performance characteristics as the backing map.
since I used ConcurrentHashMap, there is no need to synchronizedSet.
> On Sept. 14, 2012, 6:18 a.m., Aniruddha Laud wrote:
> > hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java, line 332
> > <https://reviews.apache.org/r/5086/diff/3/?file=154672#file154672line332>
> >
> > This is not necessarily true. The topic may or may not have moved.
will remove 'TOPIC_MOVED' to make log statement simple and clearly.
> On Sept. 14, 2012, 6:18 a.m., Aniruddha Laud wrote:
> > hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java, line 24
> > <https://reviews.apache.org/r/5086/diff/3/?file=154673#file154673line24>
> >
> > Please add a note stating that no blocking operations should be done in processEvent()
good point
> On Sept. 14, 2012, 6:18 a.m., Aniruddha Laud wrote:
> > hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java, line 260
> > <https://reviews.apache.org/r/5086/diff/3/?file=154680#file154680line260>
> >
> > remove topic from topic2sub2seq. Otherwise once we disconnect the subscription channel, a client-side subscription could succeed.
I added a flag in ReleaseOp to control whether remove topic2sub2seq or not. since StubSubscriptionManager in testing would use it, we should not simply remove it.
but I found that there is a bug of my code to stopDelivery in releaseTopic, since subscription states is removed from the map first, it would not be found in #finish() callback, so no subscribers found to stop. why it passed the test case I added? because in that test case, it uses StubSubscriptionManager, the states will not be removed.
I will fix this one.
- Sijie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/#review11516
-----------------------------------------------------------
On Sept. 14, 2012, 12:07 a.m., Sijie Guo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5086/
> -----------------------------------------------------------
>
> (Updated Sept. 14, 2012, 12:07 a.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
>
> client -> proxy server 1 -> hedwig
> \> proxy server 2 />
>
> when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
>
> we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
>
> the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
>
> so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
>
> when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
>
> in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
>
>
> This addresses bug BOOKKEEPER-252.
> https://issues.apache.org/jira/browse/BOOKKEEPER-252
>
>
> Diffs
> -----
>
> hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
> hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
> hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
> hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
> hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
> hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
> hedwig-client/src/main/cpp/test/util.h 45a6db3
> hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java e3d46ca
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5bcf786
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java 075705a
> hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java PRE-CREATION
> hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
> hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
> hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 7d8e22b
> hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java b9125ca
> hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
> hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
> hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
> hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
> hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
>
> Diff: https://reviews.apache.org/r/5086/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sijie Guo
>
>
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by Ivan Kelly <iv...@apache.org>.
> On Sept. 14, 2012, 6:18 a.m., Aniruddha Laud wrote:
> > hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java, line 38
> > <https://reviews.apache.org/r/5086/diff/3/?file=154673#file154673line38>
> >
> > Also pass in the channel object that was disconnected.
No, we don't want the client to get any access to hedwig's internal channels. It would completely break the encapsulation. Why do you think they need access to the channel?
> On Sept. 14, 2012, 6:18 a.m., Aniruddha Laud wrote:
> > hedwig-protocol/src/main/protobuf/PubSubProtocol.proto, line 183
> > <https://reviews.apache.org/r/5086/diff/3/?file=154675#file154675line183>
> >
> > CHANNEL_DISCONNECTED = 3 and use this event as default
See response to previous comment.
- Ivan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/#review11516
-----------------------------------------------------------
On Sept. 14, 2012, 12:07 a.m., Sijie Guo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5086/
> -----------------------------------------------------------
>
> (Updated Sept. 14, 2012, 12:07 a.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
>
> client -> proxy server 1 -> hedwig
> \> proxy server 2 />
>
> when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
>
> we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
>
> the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
>
> so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
>
> when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
>
> in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
>
>
> This addresses bug BOOKKEEPER-252.
> https://issues.apache.org/jira/browse/BOOKKEEPER-252
>
>
> Diffs
> -----
>
> hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
> hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
> hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
> hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
> hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
> hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
> hedwig-client/src/main/cpp/test/util.h 45a6db3
> hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java e3d46ca
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5bcf786
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java 075705a
> hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java PRE-CREATION
> hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
> hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
> hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 7d8e22b
> hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java b9125ca
> hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
> hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
> hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
> hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
> hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
>
> Diff: https://reviews.apache.org/r/5086/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sijie Guo
>
>
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by Aniruddha Laud <i0...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/#review11516
-----------------------------------------------------------
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java
<https://reviews.apache.org/r/5086/#comment24746>
Wrap inside a synchronizedSet() ?
hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java
<https://reviews.apache.org/r/5086/#comment24747>
This is not necessarily true. The topic may or may not have moved.
hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java
<https://reviews.apache.org/r/5086/#comment24748>
Please add a note stating that no blocking operations should be done in processEvent()
hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java
<https://reviews.apache.org/r/5086/#comment24749>
Also pass in the channel object that was disconnected.
hedwig-protocol/src/main/protobuf/PubSubProtocol.proto
<https://reviews.apache.org/r/5086/#comment24750>
CHANNEL_DISCONNECTED = 3 and use this event as default
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java
<https://reviews.apache.org/r/5086/#comment24754>
remove topic from topic2sub2seq. Otherwise once we disconnect the subscription channel, a client-side subscription could succeed.
- Aniruddha Laud
On Sept. 14, 2012, 12:07 a.m., Sijie Guo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5086/
> -----------------------------------------------------------
>
> (Updated Sept. 14, 2012, 12:07 a.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
>
> client -> proxy server 1 -> hedwig
> \> proxy server 2 />
>
> when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
>
> we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
>
> the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
>
> so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
>
> when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
>
> in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
>
>
> This addresses bug BOOKKEEPER-252.
> https://issues.apache.org/jira/browse/BOOKKEEPER-252
>
>
> Diffs
> -----
>
> hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
> hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
> hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
> hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
> hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
> hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
> hedwig-client/src/main/cpp/test/util.h 45a6db3
> hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java e3d46ca
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5bcf786
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java 075705a
> hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java PRE-CREATION
> hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
> hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
> hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 7d8e22b
> hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java b9125ca
> hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
> hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
> hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
> hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
> hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
>
> Diff: https://reviews.apache.org/r/5086/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sijie Guo
>
>
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by Sijie Guo <gu...@gmail.com>.
> On Sept. 14, 2012, 10:16 a.m., Ivan Kelly wrote:
> > hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java, line 385
> > <https://reviews.apache.org/r/5086/diff/3/?file=154670#file154670line385>
> >
> > Should be clearHostForTopic()
will fix it.
> On Sept. 14, 2012, 10:16 a.m., Ivan Kelly wrote:
> > hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java, line 111
> > <https://reviews.apache.org/r/5086/diff/3/?file=154671#file154671line111>
> >
> > why protected and not simply package private?
will fix it.
- Sijie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/#review11522
-----------------------------------------------------------
On Sept. 14, 2012, 12:07 a.m., Sijie Guo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5086/
> -----------------------------------------------------------
>
> (Updated Sept. 14, 2012, 12:07 a.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
>
> client -> proxy server 1 -> hedwig
> \> proxy server 2 />
>
> when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
>
> we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
>
> the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
>
> so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
>
> when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
>
> in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
>
>
> This addresses bug BOOKKEEPER-252.
> https://issues.apache.org/jira/browse/BOOKKEEPER-252
>
>
> Diffs
> -----
>
> hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
> hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
> hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
> hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
> hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
> hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
> hedwig-client/src/main/cpp/test/util.h 45a6db3
> hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java e3d46ca
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5bcf786
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java 075705a
> hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java PRE-CREATION
> hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
> hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
> hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 7d8e22b
> hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java b9125ca
> hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
> hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
> hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
> hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
> hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
>
> Diff: https://reviews.apache.org/r/5086/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sijie Guo
>
>
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by Ivan Kelly <iv...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/#review11522
-----------------------------------------------------------
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java
<https://reviews.apache.org/r/5086/#comment24764>
Should be clearHostForTopic()
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java
<https://reviews.apache.org/r/5086/#comment24765>
why protected and not simply package private?
- Ivan Kelly
On Sept. 14, 2012, 12:07 a.m., Sijie Guo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5086/
> -----------------------------------------------------------
>
> (Updated Sept. 14, 2012, 12:07 a.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
>
> client -> proxy server 1 -> hedwig
> \> proxy server 2 />
>
> when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
>
> we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
>
> the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
>
> so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
>
> when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
>
> in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
>
>
> This addresses bug BOOKKEEPER-252.
> https://issues.apache.org/jira/browse/BOOKKEEPER-252
>
>
> Diffs
> -----
>
> hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
> hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
> hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
> hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
> hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
> hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
> hedwig-client/src/main/cpp/test/util.h 45a6db3
> hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java e3d46ca
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5bcf786
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java 075705a
> hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java PRE-CREATION
> hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
> hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
> hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 7d8e22b
> hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java b9125ca
> hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
> hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
> hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
> hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
> hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
>
> Diff: https://reviews.apache.org/r/5086/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sijie Guo
>
>
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by Sijie Guo <gu...@gmail.com>.
> On Oct. 23, 2012, 9:14 p.m., Yixue (Andrew) Zhu wrote:
> > hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java, line 66
> > <https://reviews.apache.org/r/5086/diff/1/?file=108306#file108306line66>
> >
> > Ok. I see it is for skip reading subscriptions. It seems worth to clean it up, by tracking it in different data structure, and resurface it when reading subscription.
Yup. Agreed. It would be cleaner as you described.
- Sijie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/#review12698
-----------------------------------------------------------
On Sept. 14, 2012, 3:30 p.m., Sijie Guo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5086/
> -----------------------------------------------------------
>
> (Updated Sept. 14, 2012, 3:30 p.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
>
> client -> proxy server 1 -> hedwig
> \> proxy server 2 />
>
> when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
>
> we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
>
> the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
>
> so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
>
> when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
>
> in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
>
>
> This addresses bug BOOKKEEPER-252.
> https://issues.apache.org/jira/browse/BOOKKEEPER-252
>
>
> Diffs
> -----
>
> hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
> hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
> hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
> hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
> hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
> hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
> hedwig-client/src/main/cpp/test/util.h 45a6db3
> hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java 6730668
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5bcf786
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java c0409be
> hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java PRE-CREATION
> hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
> hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
> hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 2cc450c
> hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java b9125ca
> hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
> hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
> hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
> hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
> hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
>
> Diff: https://reviews.apache.org/r/5086/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sijie Guo
>
>
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by "Yixue (Andrew) Zhu" <yx...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/#review12698
-----------------------------------------------------------
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java
<https://reviews.apache.org/r/5086/#comment27074>
Ok. I see it is for skip reading subscriptions. It seems worth to clean it up, by tracking it in different data structure, and resurface it when reading subscription.
- Yixue (Andrew) Zhu
On Sept. 14, 2012, 3:30 p.m., Sijie Guo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5086/
> -----------------------------------------------------------
>
> (Updated Sept. 14, 2012, 3:30 p.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
>
> client -> proxy server 1 -> hedwig
> \> proxy server 2 />
>
> when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
>
> we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
>
> the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
>
> so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
>
> when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
>
> in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
>
>
> This addresses bug BOOKKEEPER-252.
> https://issues.apache.org/jira/browse/BOOKKEEPER-252
>
>
> Diffs
> -----
>
> hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
> hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
> hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
> hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
> hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
> hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
> hedwig-client/src/main/cpp/test/util.h 45a6db3
> hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java 6730668
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5bcf786
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java c0409be
> hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java PRE-CREATION
> hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
> hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
> hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 2cc450c
> hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java b9125ca
> hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
> hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
> hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
> hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
> hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
>
> Diff: https://reviews.apache.org/r/5086/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sijie Guo
>
>
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by "Yixue (Andrew) Zhu" <yx...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/#review12696
-----------------------------------------------------------
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java
<https://reviews.apache.org/r/5086/#comment27069>
Why do we keep the topic mapping in top2sub2seq by passing 'false' parameter?
I encountered conflict while preparing my patch.
- Yixue (Andrew) Zhu
On Sept. 14, 2012, 3:30 p.m., Sijie Guo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5086/
> -----------------------------------------------------------
>
> (Updated Sept. 14, 2012, 3:30 p.m.)
>
>
> Review request for bookkeeper.
>
>
> Description
> -------
>
> In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
>
> client -> proxy server 1 -> hedwig
> \> proxy server 2 />
>
> when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
>
> we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
>
> the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
>
> so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
>
> when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
>
> in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
>
>
> This addresses bug BOOKKEEPER-252.
> https://issues.apache.org/jira/browse/BOOKKEEPER-252
>
>
> Diffs
> -----
>
> hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
> hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
> hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
> hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
> hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
> hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
> hedwig-client/src/main/cpp/test/util.h 45a6db3
> hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java 6730668
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5bcf786
> hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java c0409be
> hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java PRE-CREATION
> hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
> hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
> hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 2cc450c
> hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java b9125ca
> hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
> hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
> hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
> hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
> hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
> hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
> hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
>
> Diff: https://reviews.apache.org/r/5086/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sijie Guo
>
>
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by Sijie Guo <gu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/
-----------------------------------------------------------
(Updated Sept. 14, 2012, 3:30 p.m.)
Review request for bookkeeper.
Changes
-------
addressed the comments and rebased to latest trunk.
Description
-------
In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
client -> proxy server 1 -> hedwig
\> proxy server 2 />
when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
This addresses bug BOOKKEEPER-252.
https://issues.apache.org/jira/browse/BOOKKEEPER-252
Diffs (updated)
-----
hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
hedwig-client/src/main/cpp/test/util.h 45a6db3
hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java 6730668
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5bcf786
hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java c0409be
hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java PRE-CREATION
hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 2cc450c
hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java b9125ca
hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
Diff: https://reviews.apache.org/r/5086/diff/
Testing
-------
Thanks,
Sijie Guo
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by Sijie Guo <gu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/
-----------------------------------------------------------
(Updated Sept. 14, 2012, 12:07 a.m.)
Review request for bookkeeper.
Changes
-------
update the patch to provide a client level subscription listener rather than a per-subscription listener.
Description
-------
In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
client -> proxy server 1 -> hedwig
\> proxy server 2 />
when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
This addresses bug BOOKKEEPER-252.
https://issues.apache.org/jira/browse/BOOKKEEPER-252
Diffs (updated)
-----
hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
hedwig-client/src/main/cpp/test/util.h 45a6db3
hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigClientImpl.java e3d46ca
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5bcf786
hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java 075705a
hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionListener.java PRE-CREATION
hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 7d8e22b
hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java b9125ca
hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
Diff: https://reviews.apache.org/r/5086/diff/
Testing
-------
Thanks,
Sijie Guo
Re: Review Request: BOOKKEEPER-252: Hedwig: provide a subscription mode to
kill other subscription channel when hedwig client is used as a proxy-style
server.
Posted by Sijie Guo <gu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5086/
-----------------------------------------------------------
(Updated Sept. 4, 2012, 3:34 p.m.)
Review request for bookkeeper.
Changes
-------
attach a new patch rebased to latest trunk and addressed Ivan's comments.
Description
-------
In some case, we need to hedwig-client as proxy server to provide messaging service to other users.
client -> proxy server 1 -> hedwig
\> proxy server 2 />
when client would connect to either proxy server to receive messages, the proxy server would setup subscription channel to hedwig server.
we just want client to be simple, so when the channel between client and proxy server is broken, client will try to connect to proxy servers thru VIP. it might connect to other proxy server. for example, first time client connects to proxy server 1, but the client found the connection is broken, it connects to proxy server 2. when proxy server 2 tried to setup subscription channel to hedwig, hedwig found that this subscription has existed before occupied by proxy server 1.
the panic here is that proxy server 1 only disconnect old subscription channel only when it detected the channel between client and itself is broken. The detection might be delayed due to several reasons. so it might increment the latency that messages are pushed to real client.
so we try to introduce a subscription mode called CREATE_OR_ATTACH_OR_KILL mode.
when a subscriber use this subscription mode, it would kill old existed subscription channel. when using this subscription mode, we would turn off auto-reconnect functionality in hedwig client and just tell client about the channel disconnected event so client could do its logic when channel is detected.
in order to provide some admin tool for admin guys to debug/operate, we provide ADMIN mode. if a subscriber attach to a subscription using ADMIN mode, its subscription channel would never be killed, then it is safe to guarantee admin operations.
This addresses bug BOOKKEEPER-252.
https://issues.apache.org/jira/browse/BOOKKEEPER-252
Diffs (updated)
-----
hedwig-client/src/main/cpp/inc/hedwig/callback.h 8d47018
hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 8ca5b2b
hedwig-client/src/main/cpp/lib/data.h 1539a18
hedwig-client/src/main/cpp/lib/data.cpp 9c5fb37
hedwig-client/src/main/cpp/lib/subscriberimpl.h beba547
hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 6d7506b
hedwig-client/src/main/cpp/test/subscribetest.cpp 6883867
hedwig-client/src/main/cpp/test/util.h 45a6db3
hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 880fff7
hedwig-client/src/main/java/org/apache/hedwig/client/data/PubSubData.java ce7469e
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigPublisher.java b90865f
hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 6d1ce1c
hedwig-client/src/main/java/org/apache/hedwig/client/netty/ResponseHandler.java 2b07807
hedwig-client/src/main/java/org/apache/hedwig/util/SubscriptionChannelListener.java PRE-CREATION
hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java 723eb26
hedwig-protocol/src/main/protobuf/PubSubProtocol.proto 7645c5e
hedwig-server/src/main/java/org/apache/hedwig/admin/console/HedwigConsole.java 7d8e22b
hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java 1ad9ce1
hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 7220e23
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 496817f
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/InMemorySubscriptionManager.java ed04845
hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/MMSubscriptionManager.java b7e22b5
hedwig-server/src/test/java/org/apache/hedwig/client/TestPubSubClient.java 2297c56
hedwig-server/src/test/java/org/apache/hedwig/server/PubSubServerStandAloneTestBase.java 8a61223
hedwig-server/src/test/java/org/apache/hedwig/server/handlers/TestSubUnsubHandler.java 5b53f24
hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/StubSubscriptionManager.java a9df6a5
hedwig-server/src/test/java/org/apache/hedwig/server/subscriptions/TestMMSubscriptionManager.java e06c473
Diff: https://reviews.apache.org/r/5086/diff/
Testing
-------
Thanks,
Sijie Guo