You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Aniruddha (JIRA)" <ji...@apache.org> on 2012/05/15 22:53:07 UTC
[jira] [Created] (BOOKKEEPER-261) Let a hedwig-client discover the
region of the hedwig-server cluster it is connected to
Aniruddha created BOOKKEEPER-261:
------------------------------------
Summary: Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
Key: BOOKKEEPER-261
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
Project: Bookkeeper
Issue Type: Improvement
Components: hedwig-client
Reporter: Aniruddha
Priority: Minor
Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (BOOKKEEPER-261) Let a hedwig-client discover the
region of the hedwig-server cluster it is connected to
Posted by "Aniruddha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aniruddha reassigned BOOKKEEPER-261:
------------------------------------
Assignee: Aniruddha
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Assignee: Aniruddha
> Priority: Minor
> Attachments: BK-261.patch
>
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-261) Let a hedwig-client discover
the region of the hedwig-server cluster it is connected to
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281352#comment-13281352 ]
Sijie Guo commented on BOOKKEEPER-261:
--------------------------------------
@Aniruddha, you don't need to remove those changes. the changes of PubSubProtocol.java should be included, so users don't need to use protoc to build proto file themselves.
BTW, I think it would be scheduled to release in 4.2, since we are ready to cut 4.1 version recently. Is it OK for you?
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Assignee: Aniruddha
> Priority: Minor
> Attachments: BK-261.patch
>
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-261) Let a hedwig-client discover
the region of the hedwig-server cluster it is connected to
Posted by "Aniruddha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276382#comment-13276382 ]
Aniruddha commented on BOOKKEEPER-261:
--------------------------------------
Adding this information to the response of publish/subscribe operations would 1)increase overhead for every message and 2)depend on the client performing a publish/subscribe (as the client connects lazily to the hub). How about adding an operation dedicated to getting hub information? For now it could just return the region, but this could be extended to include anything else that might be needed in future.
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Priority: Minor
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-261) Let a hedwig-client discover
the region of the hedwig-server cluster it is connected to
Posted by "Aniruddha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281304#comment-13281304 ]
Aniruddha commented on BOOKKEEPER-261:
--------------------------------------
I've attached a patch for this. The patch has the changes to the protobuf generated PubSubProtocol.java file as well. Should I remove these changes? The file is committed in git, though.
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Priority: Minor
> Attachments: BK-261.patch
>
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-261) Let a hedwig-client discover the
region of the hedwig-server cluster it is connected to
Posted by "Aniruddha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aniruddha updated BOOKKEEPER-261:
---------------------------------
Attachment: BK-261.patch
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Assignee: Aniruddha
> Priority: Minor
> Attachments: BK-261.patch
>
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-261) Let a hedwig-client discover
the region of the hedwig-server cluster it is connected to
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13282238#comment-13282238 ]
Sijie Guo commented on BOOKKEEPER-261:
--------------------------------------
Thanks Aniruddha.
some comments on your patch.
from the patch, seems that get hub info action is put as part of publisher. I don't think it is a good idea to put get hub info in publisher. actually get hub info action is a kind of admin action. How about putting it in an admin interface as below? I am not very sure is *Admin* a good name due to my poor english. maybe you could have a better name for it.
{code}
public interface Admin {
public String getRegion();
}
public interface Client {
public Admin getAdmin();
}
{code}
besides that hubInfo is a kind of Nouns, not a verb, while publish/subscribe are verbs. So I would suggest changing it to 'GetHubInfo'.
HedwigClientImpl:
why return null instead of throwing exception? It would be easy to client to know what happened than hiding it.
HubInfoHandler:
it would be better to include Apache license in the header of the file.
{quote}
String infoRequest = cfg.getMyRegion();
{quote}
it would be better to have a meaningful name more than 'infoRequest'.
BTW, it would be better that if you could put the patch on review board, which might be easy for reviewing. :) https://reviews.apache.org/dashboard/
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Assignee: Aniruddha
> Priority: Minor
> Attachments: BK-261.patch
>
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-261) Let a hedwig-client discover
the region of the hedwig-server cluster it is connected to
Posted by "Aniruddha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13282167#comment-13282167 ]
Aniruddha commented on BOOKKEEPER-261:
--------------------------------------
That sounds fine Sijie.
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Assignee: Aniruddha
> Priority: Minor
> Attachments: BK-261.patch
>
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-261) Let a hedwig-client discover
the region of the hedwig-server cluster it is connected to
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276393#comment-13276393 ]
Sijie Guo commented on BOOKKEEPER-261:
--------------------------------------
yes. the overhead would be an issue for publish requests, for subscribe, it is OK.
I misunderstood you proposal 2) before. What you want is the hub information who is the owner of a topic. A separated operations to get owner info of a topic would be better. This operation would leverage same channel with publish. seems OK from my side.
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Priority: Minor
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-261) Let a hedwig-client discover
the region of the hedwig-server cluster it is connected to
Posted by "Aniruddha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276351#comment-13276351 ]
Aniruddha commented on BOOKKEEPER-261:
--------------------------------------
One possible use case is when we might have a number of Clients in one process, each connected to a server in a different region. It would be useful to get the connected region information without having to include it as a configuration parameter.
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Priority: Minor
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-261) Let a hedwig-client discover
the region of the hedwig-server cluster it is connected to
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276374#comment-13276374 ]
Sijie Guo commented on BOOKKEEPER-261:
--------------------------------------
@Aniruddha, thanks for explanation.
the case is OK, but just be curious that it seems that all the clients connect to a same domain name and which host the client would connect is determined according to some kind of geo-location. If so, there is a chance that a subscribe is moved from one region to another region. You might encounter inconsistent subscription state problem during subscriber movement as described in BOOKKEEPER-147.
If it worths discovering the region info, I prefer 2) solution you proposed. Because hub server has the information which region it belongs to, you just need to add the region info at the response of publish/subscribe. And I would suggest it could be done more flexible by adding a meta block (a key/value map) in the response of publish/subscribe like http response header. 1) region info could be respond as the value of meta key 'Region' 2) it might be easy for us to add more in future if necessary.
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Priority: Minor
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-261) Let a hedwig-client discover
the region of the hedwig-server cluster it is connected to
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276343#comment-13276343 ]
Sijie Guo commented on BOOKKEEPER-261:
--------------------------------------
If we treat Hedwig as a whole system, the publish/subscribe should just connect to its local region, cross-region functionality would be handled by Hedwig itself. From this side, the client might know which region it connected.
so one quick question, why you need the region for hedwig client? is there any special considerations?
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Priority: Minor
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (BOOKKEEPER-261) Let a hedwig-client discover the
region of the hedwig-server cluster it is connected to
Posted by "Aniruddha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aniruddha resolved BOOKKEEPER-261.
----------------------------------
Resolution: Duplicate
Marked this as a duplicate. See https://issues.apache.org/jira/browse/BOOKKEEPER-323
> Let a hedwig-client discover the region of the hedwig-server cluster it is connected to
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-261
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-261
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client
> Reporter: Aniruddha
> Assignee: Aniruddha
> Priority: Minor
> Attachments: BK-261.patch
>
>
> Currently, the client-server protocol doesn't let a Hedwig client discover the region that it is a part of. There are a couple of approaches that I could think of to do this - 1) The client publishes and subscribes to a randomly generated topic-name. It can then set it's region to the srcRegion of the received message. 2) Have a discovery operation akin to the publish/subscribe operations and let the server respond to this with information about itself (region name and any other information it might want to pass on to the client). I personally prefer (2). I could produce a patch if the approach looks good.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira