You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Yasin <ya...@gmail.com> on 2013/01/15 17:23:16 UTC

Implementing DNS-like service using ZooKeeper

Hi all,

I am relatively new to Zookeeper. I run it as a single standalone server on
my laptop and tried it with 3 server-cluster on a private cloud. And tried
some sample codes. 

I want to develop a service acts like a DNS. It will basically keep IPs of
thousands of host along with some additional information such as Rack ID.
Let's call this service zkDNS. I am at the design phase. I want design the
system as good as possible so that I will not have to make so too much
changes on the way. I have a couple of questions:

1- Can I run zkDNS and ZooKeeper on the same physical machines at the same
time? This will give me localization advantage. zkDNS process will connect
to ZooKeeper service residing on the same machine. There will not be network
traffic.

2- zkDNS will store maybe hundreds of thousands of IP addresses and will
serve these hosts asking for some information about other hosts. How should
I implement this schema? Should I keep IP addresses in ArrayLists and create
a znode on Zookeeper for every IP address? This does not seem to be a good
design. I will need to search the zkDNS host list for every request.
Therefore I want to keep IP addresses in sorted ordered. 

How can I start this project by taking advantages of ZooKeeper service? 
I need a starting point/approach the problem.
Any idea/comment/link is appreciated.

Best

Yasin



--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Implementing-DNS-like-service-using-ZooKeeper-tp7578405.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

Re: Implementing DNS-like service using ZooKeeper

Posted by Matt Wise <ma...@nextdoor.com>.
Curator is excellent at this. Additionally, zk_watcher is a simple way of doing this as well.

http://pypi.python.org/pypi/zk_watcher

(for what its worth, we do almost exactly this at Nextdoor... every server we have runs this daemon, and they register various bits of data about themselves in a Zookeeper path. We then leverage this path with other tools for getting full server-lists.)

--Matt

On Jan 15, 2013, at 10:33 AM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:

> FYI. Curator provides a Service Discovery implementation:
> 
> https://github.com/Netflix/curator/wiki/Service-Discovery
> 
> ====================
> Jordan Zimmerman
> 
> On Jan 15, 2013, at 10:19 AM, Yasin <ya...@gmail.com> wrote:
> 
> Hi all,
> 
> I am relatively new to Zookeeper. I run it as a single standalone server on
> my laptop and tried it with 3 server-cluster on a private cloud. And tried
> some sample codes.
> 
> I want to develop a service acts like a DNS. It will basically keep IPs of
> thousands of host along with some additional information such as Rack ID.
> Let's call this service zkDNS. I am at the design phase. I want design the
> system as good as possible so that I will not have to make so too much
> changes on the way. I have a couple of questions:
> 
> 1- Can I run zkDNS and ZooKeeper on the same physical machines at the same
> time? This will give me localization advantage. zkDNS process will connect
> to ZooKeeper service residing on the same machine. There will not be network
> traffic.
> 
> 2- zkDNS will store maybe hundreds of thousands of IP addresses and will
> serve these hosts asking for some information about other hosts. How should
> I implement this schema? Should I keep IP addresses in ArrayLists and create
> a znode on Zookeeper for every IP address? This does not seem to be a good
> design. I will need to search the zkDNS host list for every request.
> Therefore I want to keep IP addresses in sorted ordered.
> 
> How can I start this project by taking advantages of ZooKeeper service?
> I need a starting point/approach the problem.
> Any idea/comment/link is appreciated.
> 
> Best
> 
> Yasin
> 
> 
> 
> --
> View this message in context:
> http://zookeeper-user.578899.n2.nabble.com/Implementing-DNS-like-service-using-ZooKeeper-tp7578405.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.

Re: Implementing DNS-like service using ZooKeeper

Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
FYI. Curator provides a Service Discovery implementation:

https://github.com/Netflix/curator/wiki/Service-Discovery

====================
Jordan Zimmerman

On Jan 15, 2013, at 10:19 AM, Yasin <ya...@gmail.com> wrote:

Hi all,

I am relatively new to Zookeeper. I run it as a single standalone server on
my laptop and tried it with 3 server-cluster on a private cloud. And tried
some sample codes.

I want to develop a service acts like a DNS. It will basically keep IPs of
thousands of host along with some additional information such as Rack ID.
Let's call this service zkDNS. I am at the design phase. I want design the
system as good as possible so that I will not have to make so too much
changes on the way. I have a couple of questions:

1- Can I run zkDNS and ZooKeeper on the same physical machines at the same
time? This will give me localization advantage. zkDNS process will connect
to ZooKeeper service residing on the same machine. There will not be network
traffic.

2- zkDNS will store maybe hundreds of thousands of IP addresses and will
serve these hosts asking for some information about other hosts. How should
I implement this schema? Should I keep IP addresses in ArrayLists and create
a znode on Zookeeper for every IP address? This does not seem to be a good
design. I will need to search the zkDNS host list for every request.
Therefore I want to keep IP addresses in sorted ordered.

How can I start this project by taking advantages of ZooKeeper service?
I need a starting point/approach the problem.
Any idea/comment/link is appreciated.

Best

Yasin



--
View this message in context:
http://zookeeper-user.578899.n2.nabble.com/Implementing-DNS-like-service-using-ZooKeeper-tp7578405.html
Sent from the zookeeper-user mailing list archive at Nabble.com.