You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Check Peck <co...@gmail.com> on 2014/04/07 03:46:52 UTC
Re: Load on Zookeeper cluster?
Thanks guys.. I am going to do load test in couple of days using the
exiting Python framework. And I will share the results once I am done.
On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R <ra...@huawei.com> wrote:
>
> If I understood the case correctly,
> Test :- 3 node ZK cluster, 600 sessions each will do one read
> operation(getChildren call) every one minute.
>
> I'm assuming the calls are equally distributed in time frame and it would
> be like : 600ops/60secs, average 10 ops/sec will go to 3 node cluster
> I also feel it should be, also it would be great if you publish the
> results after your tests.
>
>
> -Rakesh
>
> -----Original Message-----
> From: Check Peck [mailto:comptechgeeky@gmail.com]
> Sent: 26 March 2014 11:19
> To: user
> Subject: Re: Load on Zookeeper cluster?
>
> Thanks that is also quite useful.
>
> For my above use case, I just need to get the children of a particular
> znode.. And that's what will be happening from all the 600 application
> machines..
>
> From the wiki you provided, it looks it should be fine right?
>
>
> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R <ra...@huawei.com> wrote:
>
> >
> > Please go through the below one, hope it will be helpful to you. Also,
> > I would recommend to have a look at the basic zk configurations too.
> > All the
> > best:)
> >
> >
> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
> > erview
> >
> >
> > What client operations(for ex: simply creating many znodes, deleting,
> > setwatches et.) you are planning to perform.
> > Actually I'd like to know what each zkclient will be doing.
> >
> > -Rakesh
> >
> > -----Original Message-----
> > From: Check Peck [mailto:comptechgeeky@gmail.com]
> > Sent: 26 March 2014 09:40
> > To: user
> > Subject: Re: Load on Zookeeper cluster?
> >
> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés <
> > rgs@itevenworks.net
> > > wrote:
> >
> > > 1 ZK se
> >
> >
> >
> > Yes one ZK session per application. Zookeeper version I am running
> > currently is 3.4.5 in production.
> >
>
Re: Load on Zookeeper cluster?
Posted by Check Peck <co...@gmail.com>.
How would I do this then? Is there any existing frameworks which I can use
or I need to write my own code for this?
On Tue, Apr 8, 2014 at 8:54 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>wrote:
> Ok maybe you should try exactly that instead of using zk-latency.py.
> Create 600 sessions and do a get once every minute. ZooKeeper should
> be able to handle that.
>
> On Tue, Apr 8, 2014 at 7:05 PM, Check Peck <co...@gmail.com>
> wrote:
> > Actual scenario will be -
> >
> >
> > Test :- 3 node ZK cluster, 600 sessions each will do one read
> > operation(getChildren call) every one minute.
> >
> > At what configuration I should try it out.
> >
> >
> > On Tue, Apr 8, 2014 at 6:25 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>
> > wrote:
> >>
> >> > So my question is what does this line mean at the end? Does it say, it
> >> > can
> >> > do a get request call at a rate of 2557 requests per second?
> >> >
> >> > get 100 znodes in 39 ms (0.390978 ms/op
> >> > 2557.689587/sec)
> >>
> >> Yes.
> >>
> >> > Also I was interested to see how much load a 3 node ZK cluster can
> >> > handle.
> >> > In my case, most of the call will be "get" (get the data from only one
> >> > znode) on the ZK cluster So with the above test I should be able to
> get
> >> > some
> >> > idea right?
> >>
> >> How many ZK clients do you expect to have? I would run the same number
> >> of zk-latency.py instances with higher number of operations to get a
> >> more accurate picture. The performance also depends on whether you use
> >> asynchronous reads or synchronous reads.
> >
> >
>
Re: Load on Zookeeper cluster?
Posted by Michi Mutsuzaki <mi...@cs.stanford.edu>.
Ok maybe you should try exactly that instead of using zk-latency.py.
Create 600 sessions and do a get once every minute. ZooKeeper should
be able to handle that.
On Tue, Apr 8, 2014 at 7:05 PM, Check Peck <co...@gmail.com> wrote:
> Actual scenario will be -
>
>
> Test :- 3 node ZK cluster, 600 sessions each will do one read
> operation(getChildren call) every one minute.
>
> At what configuration I should try it out.
>
>
> On Tue, Apr 8, 2014 at 6:25 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>
> wrote:
>>
>> > So my question is what does this line mean at the end? Does it say, it
>> > can
>> > do a get request call at a rate of 2557 requests per second?
>> >
>> > get 100 znodes in 39 ms (0.390978 ms/op
>> > 2557.689587/sec)
>>
>> Yes.
>>
>> > Also I was interested to see how much load a 3 node ZK cluster can
>> > handle.
>> > In my case, most of the call will be "get" (get the data from only one
>> > znode) on the ZK cluster So with the above test I should be able to get
>> > some
>> > idea right?
>>
>> How many ZK clients do you expect to have? I would run the same number
>> of zk-latency.py instances with higher number of operations to get a
>> more accurate picture. The performance also depends on whether you use
>> asynchronous reads or synchronous reads.
>
>
Re: Load on Zookeeper cluster?
Posted by Check Peck <co...@gmail.com>.
Actual scenario will be -
Test :- 3 node ZK cluster, 600 sessions each will do one read
operation(getChildren call) every one minute.
At what configuration I should try it out.
On Tue, Apr 8, 2014 at 6:25 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>wrote:
> > So my question is what does this line mean at the end? Does it say, it
> can
> > do a get request call at a rate of 2557 requests per second?
> >
> > get 100 znodes in 39 ms (0.390978 ms/op
> > 2557.689587/sec)
>
> Yes.
>
> > Also I was interested to see how much load a 3 node ZK cluster can
> handle.
> > In my case, most of the call will be "get" (get the data from only one
> > znode) on the ZK cluster So with the above test I should be able to get
> some
> > idea right?
>
> How many ZK clients do you expect to have? I would run the same number
> of zk-latency.py instances with higher number of operations to get a
> more accurate picture. The performance also depends on whether you use
> asynchronous reads or synchronous reads.
>
Re: Load on Zookeeper cluster?
Posted by Michi Mutsuzaki <mi...@cs.stanford.edu>.
> So my question is what does this line mean at the end? Does it say, it can
> do a get request call at a rate of 2557 requests per second?
>
> get 100 znodes in 39 ms (0.390978 ms/op
> 2557.689587/sec)
Yes.
> Also I was interested to see how much load a 3 node ZK cluster can handle.
> In my case, most of the call will be "get" (get the data from only one
> znode) on the ZK cluster So with the above test I should be able to get some
> idea right?
How many ZK clients do you expect to have? I would run the same number
of zk-latency.py instances with higher number of operations to get a
more accurate picture. The performance also depends on whether you use
asynchronous reads or synchronous reads.
Re: Load on Zookeeper cluster?
Posted by Check Peck <co...@gmail.com>.
Thanks a lot Michi. Yes that works like a charm. Appreciated your help.
Now coming back to my original question as I am interested in doing load
testing on my 3 node ZK cluster. So I ran "zk-latencies.py" script to do
the load test and this is what I got on my screen -
check@phx5tg03c-cfc7:~/zooksmoketesting$ PYTHONPATH=lib.linux-i686-2.6
LD_LIBRARY_PATH=lib.linux-x86_64-2.6 ./zk-latencies.py --servers
"host1:2181,host2:2181,host3:2181" --znode_count=100 --znode_size=100
--synchronous
Connecting to host1:2181
Connected in 547 ms, handle is 0
Connecting to host2:2181
Connected in 3 ms, handle is 1
Connecting to host3:2181
Connected in 8 ms, handle is 2
Testing latencies on server host1:2181 using syncronous calls
created 100 permanent znodes in 117 ms (1.178288 ms/op
848.688614/sec)
set 100 znodes in 112 ms (1.129220 ms/op
885.567396/sec)
get 100 znodes in 39 ms (0.390978 ms/op
2557.689587/sec)
deleted 100 permanent znodes in 107 ms (1.071839 ms/op
932.975652/sec)
created 100 ephemeral znodes in 111 ms (1.118169 ms/op
894.319330/sec)
watched 100 znodes in 35 ms (0.354149 ms/op
2823.667540/sec)
deleted 100 ephemeral znodes in 110 ms (1.102810 ms/op
906.774590/sec)
notif 100 watches in 1000 ms (10.007010 ms/op
99.929954/sec)
Testing latencies on server host2:2181 using syncronous calls
created 100 permanent znodes in 108 ms (1.088130 ms/op
919.007439/sec)
set 100 znodes in 104 ms (1.049511 ms/op
952.824304/sec)
get 100 znodes in 27 ms (0.277209 ms/op
3607.382816/sec)
deleted 100 permanent znodes in 133 ms (1.330690 ms/op
751.489612/sec)
created 100 ephemeral znodes in 102 ms (1.021519 ms/op
978.934593/sec)
watched 100 znodes in 32 ms (0.327361 ms/op
3054.735079/sec)
deleted 100 ephemeral znodes in 110 ms (1.109312 ms/op
901.459984/sec)
notif 100 watches in 1001 ms (10.010960 ms/op
99.890519/sec)
Testing latencies on server host3:2181 using syncronous calls
created 100 permanent znodes in 87 ms (0.874031 ms/op
1144.124715/sec)
set 100 znodes in 85 ms (0.854771 ms/op
1169.903798/sec)
get 100 znodes in 33 ms (0.338831 ms/op
2951.324270/sec)
deleted 100 permanent znodes in 81 ms (0.818131 ms/op
1222.298185/sec)
created 100 ephemeral znodes in 81 ms (0.814011 ms/op
1228.484481/sec)
watched 100 znodes in 27 ms (0.279410 ms/op
3578.971440/sec)
deleted 100 ephemeral znodes in 99 ms (0.997062 ms/op
1002.946929/sec)
notif 100 watches in 1001 ms (10.011010 ms/op
99.890019/sec)
Latency test complete
So my question is what does this line mean at the end? Does it say, it can
do a get request call at a rate of 2557 requests per second?
get 100 znodes in 39 ms (0.390978 ms/op
2557.689587/sec)
Also I was interested to see how much load a 3 node ZK cluster can handle.
In my case, most of the call will be "get" (get the data from only one
znode) on the ZK cluster So with the above test I should be able to get
some idea right?
On Mon, Apr 7, 2014 at 11:40 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>wrote:
> Does this work?
>
> https://paste.apache.org/9Op5?action=download
>
> On Mon, Apr 7, 2014 at 9:10 PM, Check Peck <co...@gmail.com>
> wrote:
> > I manually copied the jar file into the directory it was trying to copy.
> Now
> > if I do "ant", then I see like this -
> >
> > check@phx5tg03c-cfc7:~/zooksmoketest/zookeeper-3.4.6$ ant
> > Buildfile: /home/check/zooksmoketest/zookeeper-3.4.6/build.xml
> >
> > init:
> >
> > ivy-download:
> >
> > ivy-taskdef:
> >
> > ivy-init:
> >
> > ivy-retrieve:
> > [ivy:retrieve] :: Ivy 2.2.0 - 20100923230623 ::
> http://ant.apache.org/ivy/
> > ::
> > [ivy:retrieve] :: loading settings :: file =
> > /home/check/zooksmoketest/zookeeper-3.4.6/ivysettings.xml
> > [ivy:retrieve] :: resolving dependencies ::
> > org.apache.zookeeper#zookeeper;3.4.6
> > [ivy:retrieve] confs: [default]
> >
> >
> > And then after this, it doesn't move further, I waited for a long time
> and
> > nothing came up.. And afterwards I did ctrl + c to break.
> >
> > Any idea what might be wrong?
> >
> >
> >
> > On Mon, Apr 7, 2014 at 8:42 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>
> > wrote:
> >>
> >> You probably need to set the proxy.
> >>
> >> http://ant.apache.org/manual/proxy.html
> >>
> >> On Mon, Apr 7, 2014 at 8:40 PM, Check Peck <co...@gmail.com>
> >> wrote:
> >> > Thanks a lot Michi. Appreciated your help.. I tried your suggestion
> on
> >> > one
> >> > of my production box but when I tried "fifth" step which is "ant". I
> got
> >> > an
> >> > error as shown below. Any thoughts what is wrong? I guess firewall is
> >> > blocking. Is there any other way around for this? I manually copied
> >> > "zookeeper-3.4.6.tar.gz" into that machine using "scp" as well.
> >> > check@phx5tg03c-cfc7:~/zooksmoketest/zookeeper-3.4.6$ ant
> >> > Buildfile: /home/check/zooksmoketest/zookeeper-3.4.6/build.xml
> >> >
> >> > init:
> >> >
> >> > ivy-download:
> >> > [get] Getting:
> >> > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
> >> > [get] To:
> >> > /home/check/zooksmoketest/zookeeper-3.4.6/src/java/lib/ivy-2.2.0.jar
> >> > [get] Error getting
> >> > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jarto
> >> > /home/check/zooksmoketest/zookeeper-3.4.6/src/java/lib/ivy-2.2.0.jar
> >> >
> >> > BUILD FAILED
> >> > /home/check/zooksmoketest/zookeeper-3.4.6/build.xml:348:
> >> > java.net.ConnectException: Connection timed out
> >> > at java.net.PlainSocketImpl.socketConnect(Native Method)
> >> > at
> >> >
> >> >
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
> >> > at
> >> >
> >> >
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
> >> > at
> >> >
> >> >
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
> >> > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
> >> > at java.net.Socket.connect(Socket.java:546)
> >> > at java.net.Socket.connect(Socket.java:495)
> >> > at sun.net.NetworkClient.doConnect(NetworkClient.java:178)
> >> > at sun.net.www.http.HttpClient.openServer(HttpClient.java:409)
> >> > at sun.net.www.http.HttpClient.openServer(HttpClient.java:530)
> >> > at sun.net.www.http.HttpClient.<init>(HttpClient.java:240)
> >> > at sun.net.www.http.HttpClient.New(HttpClient.java:321)
> >> > at sun.net.www.http.HttpClient.New(HttpClient.java:338)
> >> > at
> >> >
> >> >
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:935)
> >> > at
> >> >
> >> >
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:876)
> >> > at
> >> >
> >> >
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:801)
> >> > at
> >> >
> org.apache.tools.ant.taskdefs.Get$GetThread.openConnection(Get.java:660)
> >> > at
> org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:579)
> >> > at
> org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569)
> >> >
> >> > Total time: 1 minute 3 seconds
> >> >
> >> >
> >> >
> >> > On Mon, Apr 7, 2014 at 3:08 PM, Michi Mutsuzaki <
> michi@cs.stanford.edu>
> >> > wrote:
> >> >>
> >> >> Hi Check,
> >> >>
> >> >> Something like this should work on ubuntu.
> >> >>
> >> >> https://paste.apache.org/6kao?action=download
> >> >>
> >> >>
> >> >> On Sun, Apr 6, 2014 at 11:11 PM, Check Peck <comptechgeeky@gmail.com
> >
> >> >> wrote:
> >> >> > I am not sure from where to start to install zkpython.. I tried
> >> >> > following
> >> >> > that github link but was not able to understand. If you can guide
> me
> >> >> > what I
> >> >> > am supposed to do step by step, then it will be of great help to
> me.
> >> >> > Thanks.
> >> >> >
> >> >> >
> >> >> > On Sun, Apr 6, 2014 at 10:57 PM, Pavan Sudheendra
> >> >> > <pa...@gmail.com>wrote:
> >> >> >
> >> >> >> Hi Check,
> >> >> >>
> >> >> >> Where are you getting stuck?
> >> >> >>
> >> >> >>
> >> >> >> On Mon, Apr 7, 2014 at 11:13 AM, Check Peck
> >> >> >> <co...@gmail.com>
> >> >> >> wrote:
> >> >> >>
> >> >> >> > Thanks. I tried going through that github link but wasn't able
> to
> >> >> >> > understand what I am supposed to do clearly. May be I am not
> that
> >> >> >> > much
> >> >> >> > familiar with Python so that's why I am having hard time
> figuring
> >> >> >> > that
> >> >> >> out?
> >> >> >> >
> >> >> >> >
> >> >> >> > On Sun, Apr 6, 2014 at 10:06 PM, Michi Mutsuzaki
> >> >> >> > <michi@cs.stanford.edu
> >> >> >> > >wrote:
> >> >> >> >
> >> >> >> > > You need to install zkpython. The steps are described in the
> >> >> >> > > README
> >> >> >> > > file. Let us know if you have trouble installing zkpython.
> >> >> >> > >
> >> >> >> > >
> >> >> >> >
> >> >> >>
> >> >> >>
> >> >> >>
> https://github.com/apache/zookeeper/blob/trunk/src/contrib/zkpython/README
> >> >> >> > >
> >> >> >> > > On Sun, Apr 6, 2014 at 9:21 PM, Check Peck
> >> >> >> > > <co...@gmail.com>
> >> >> >> > > wrote:
> >> >> >> > > > I tried following the steps mentioned on this github link -
> >> >> >> > > > https://github.com/phunt/zk-smoketest for zk-latencies.py.
> >> >> >> > > >
> >> >> >> > > > I have 3 node ZK cluster- machineA, machineB, machineC
> >> >> >> > > >
> >> >> >> > > > Now I copied the Python program zk-latencies.py and
> >> >> >> > > > zkclient.py
> >> >> >> > > > in
> >> >> >> > > another
> >> >> >> > > > machineD in which no zookeeper is running. It's a plain
> ubuntu
> >> >> >> > > > box.
> >> >> >> > > >
> >> >> >> > > > Now when I ran the program like this -
> >> >> >> > > >
> >> >> >> > > > david@machineD:~/zooksmoketest$ ./zk-latencies.py --servers
> >> >> >> > > > "machineA:2181,machineB::2181,machineC:2181"
> --znode_count=100
> >> >> >> > > > --znode_size=100 --synchronous
> >> >> >> > > > Traceback (most recent call last):
> >> >> >> > > > File "./zk-latencies.py", line 22, in <module>
> >> >> >> > > > import zkclient
> >> >> >> > > > File "/home/david/zooksmoketest/zkclient.py", line 17, in
> >> >> >> > > > <module>
> >> >> >> > > > import zookeeper, time, threading
> >> >> >> > > >
> >> >> >> > > > *ImportError: No module named zookeeper*
> >> >> >> > > >
> >> >> >> > > > I got an error as shown above. Any thoughts what wrong I am
> >> >> >> > > > doing
> >> >> >> > here? I
> >> >> >> > > > don't have that much experience with Python. Any suggestions
> >> >> >> > > > will
> >> >> >> > > > be
> >> >> >> of
> >> >> >> > > > great help.
> >> >> >> > > >
> >> >> >> > > > @Raul: Any thoughts?
> >> >> >> > > >
> >> >> >> > > >
> >> >> >> > > >
> >> >> >> > > > On Sun, Apr 6, 2014 at 6:46 PM, Check Peck
> >> >> >> > > > <co...@gmail.com>
> >> >> >> > > wrote:
> >> >> >> > > >
> >> >> >> > > >> Thanks guys.. I am going to do load test in couple of days
> >> >> >> > > >> using
> >> >> >> > > >> the
> >> >> >> > > >> exiting Python framework. And I will share the results
> once I
> >> >> >> > > >> am
> >> >> >> done.
> >> >> >> > > >>
> >> >> >> > > >>
> >> >> >> > > >> On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R
> >> >> >> > > >> <ra...@huawei.com>
> >> >> >> > wrote:
> >> >> >> > > >>
> >> >> >> > > >>>
> >> >> >> > > >>> If I understood the case correctly,
> >> >> >> > > >>> Test :- 3 node ZK cluster, 600 sessions each will do one
> >> >> >> > > >>> read
> >> >> >> > > >>> operation(getChildren call) every one minute.
> >> >> >> > > >>>
> >> >> >> > > >>> I'm assuming the calls are equally distributed in time
> frame
> >> >> >> > > >>> and it
> >> >> >> > > would
> >> >> >> > > >>> be like : 600ops/60secs, average 10 ops/sec will go to 3
> >> >> >> > > >>> node
> >> >> >> cluster
> >> >> >> > > >>> I also feel it should be, also it would be great if you
> >> >> >> > > >>> publish
> >> >> >> > > >>> the
> >> >> >> > > >>> results after your tests.
> >> >> >> > > >>>
> >> >> >> > > >>>
> >> >> >> > > >>> -Rakesh
> >> >> >> > > >>>
> >> >> >> > > >>> -----Original Message-----
> >> >> >> > > >>> From: Check Peck [mailto:comptechgeeky@gmail.com]
> >> >> >> > > >>> Sent: 26 March 2014 11:19
> >> >> >> > > >>> To: user
> >> >> >> > > >>> Subject: Re: Load on Zookeeper cluster?
> >> >> >> > > >>>
> >> >> >> > > >>> Thanks that is also quite useful.
> >> >> >> > > >>>
> >> >> >> > > >>> For my above use case, I just need to get the children of
> a
> >> >> >> > particular
> >> >> >> > > >>> znode.. And that's what will be happening from all the 600
> >> >> >> > application
> >> >> >> > > >>> machines..
> >> >> >> > > >>>
> >> >> >> > > >>> From the wiki you provided, it looks it should be fine
> >> >> >> > > >>> right?
> >> >> >> > > >>>
> >> >> >> > > >>>
> >> >> >> > > >>> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R
> >> >> >> > > >>> <ra...@huawei.com>
> >> >> >> > wrote:
> >> >> >> > > >>>
> >> >> >> > > >>> >
> >> >> >> > > >>> > Please go through the below one, hope it will be helpful
> >> >> >> > > >>> > to
> >> >> >> > > >>> > you.
> >> >> >> > > Also,
> >> >> >> > > >>> > I would recommend to have a look at the basic zk
> >> >> >> > > >>> > configurations
> >> >> >> > too.
> >> >> >> > > >>> > All the
> >> >> >> > > >>> > best:)
> >> >> >> > > >>> >
> >> >> >> > > >>> >
> >> >> >> > > >>> >
> >> >> >> > >
> >> >> >> > >
> >> >> >> > >
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
> >> >> >> > > >>> > erview
> >> >> >> > > >>> >
> >> >> >> > > >>> >
> >> >> >> > > >>> > What client operations(for ex: simply creating many
> >> >> >> > > >>> > znodes,
> >> >> >> > deleting,
> >> >> >> > > >>> > setwatches et.) you are planning to perform.
> >> >> >> > > >>> > Actually I'd like to know what each zkclient will be
> >> >> >> > > >>> > doing.
> >> >> >> > > >>> >
> >> >> >> > > >>> > -Rakesh
> >> >> >> > > >>> >
> >> >> >> > > >>> > -----Original Message-----
> >> >> >> > > >>> > From: Check Peck [mailto:comptechgeeky@gmail.com]
> >> >> >> > > >>> > Sent: 26 March 2014 09:40
> >> >> >> > > >>> > To: user
> >> >> >> > > >>> > Subject: Re: Load on Zookeeper cluster?
> >> >> >> > > >>> >
> >> >> >> > > >>> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés
> <
> >> >> >> > > >>> > rgs@itevenworks.net
> >> >> >> > > >>> > > wrote:
> >> >> >> > > >>> >
> >> >> >> > > >>> > > 1 ZK se
> >> >> >> > > >>> >
> >> >> >> > > >>> >
> >> >> >> > > >>> >
> >> >> >> > > >>> > Yes one ZK session per application. Zookeeper version I
> am
> >> >> >> running
> >> >> >> > > >>> > currently is 3.4.5 in production.
> >> >> >> > > >>> >
> >> >> >> > > >>>
> >> >> >> > > >>
> >> >> >> > > >>
> >> >> >> > >
> >> >> >> >
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Regards-
> >> >> >> Pavan
> >> >> >>
> >> >
> >> >
> >
> >
>
Re: Load on Zookeeper cluster?
Posted by Michi Mutsuzaki <mi...@cs.stanford.edu>.
Does this work?
https://paste.apache.org/9Op5?action=download
On Mon, Apr 7, 2014 at 9:10 PM, Check Peck <co...@gmail.com> wrote:
> I manually copied the jar file into the directory it was trying to copy. Now
> if I do "ant", then I see like this -
>
> check@phx5tg03c-cfc7:~/zooksmoketest/zookeeper-3.4.6$ ant
> Buildfile: /home/check/zooksmoketest/zookeeper-3.4.6/build.xml
>
> init:
>
> ivy-download:
>
> ivy-taskdef:
>
> ivy-init:
>
> ivy-retrieve:
> [ivy:retrieve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/
> ::
> [ivy:retrieve] :: loading settings :: file =
> /home/check/zooksmoketest/zookeeper-3.4.6/ivysettings.xml
> [ivy:retrieve] :: resolving dependencies ::
> org.apache.zookeeper#zookeeper;3.4.6
> [ivy:retrieve] confs: [default]
>
>
> And then after this, it doesn't move further, I waited for a long time and
> nothing came up.. And afterwards I did ctrl + c to break.
>
> Any idea what might be wrong?
>
>
>
> On Mon, Apr 7, 2014 at 8:42 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>
> wrote:
>>
>> You probably need to set the proxy.
>>
>> http://ant.apache.org/manual/proxy.html
>>
>> On Mon, Apr 7, 2014 at 8:40 PM, Check Peck <co...@gmail.com>
>> wrote:
>> > Thanks a lot Michi. Appreciated your help.. I tried your suggestion on
>> > one
>> > of my production box but when I tried "fifth" step which is "ant". I got
>> > an
>> > error as shown below. Any thoughts what is wrong? I guess firewall is
>> > blocking. Is there any other way around for this? I manually copied
>> > "zookeeper-3.4.6.tar.gz" into that machine using "scp" as well.
>> > check@phx5tg03c-cfc7:~/zooksmoketest/zookeeper-3.4.6$ ant
>> > Buildfile: /home/check/zooksmoketest/zookeeper-3.4.6/build.xml
>> >
>> > init:
>> >
>> > ivy-download:
>> > [get] Getting:
>> > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
>> > [get] To:
>> > /home/check/zooksmoketest/zookeeper-3.4.6/src/java/lib/ivy-2.2.0.jar
>> > [get] Error getting
>> > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar to
>> > /home/check/zooksmoketest/zookeeper-3.4.6/src/java/lib/ivy-2.2.0.jar
>> >
>> > BUILD FAILED
>> > /home/check/zooksmoketest/zookeeper-3.4.6/build.xml:348:
>> > java.net.ConnectException: Connection timed out
>> > at java.net.PlainSocketImpl.socketConnect(Native Method)
>> > at
>> >
>> > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
>> > at
>> >
>> > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
>> > at
>> >
>> > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
>> > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
>> > at java.net.Socket.connect(Socket.java:546)
>> > at java.net.Socket.connect(Socket.java:495)
>> > at sun.net.NetworkClient.doConnect(NetworkClient.java:178)
>> > at sun.net.www.http.HttpClient.openServer(HttpClient.java:409)
>> > at sun.net.www.http.HttpClient.openServer(HttpClient.java:530)
>> > at sun.net.www.http.HttpClient.<init>(HttpClient.java:240)
>> > at sun.net.www.http.HttpClient.New(HttpClient.java:321)
>> > at sun.net.www.http.HttpClient.New(HttpClient.java:338)
>> > at
>> >
>> > sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:935)
>> > at
>> >
>> > sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:876)
>> > at
>> >
>> > sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:801)
>> > at
>> > org.apache.tools.ant.taskdefs.Get$GetThread.openConnection(Get.java:660)
>> > at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:579)
>> > at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569)
>> >
>> > Total time: 1 minute 3 seconds
>> >
>> >
>> >
>> > On Mon, Apr 7, 2014 at 3:08 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>
>> > wrote:
>> >>
>> >> Hi Check,
>> >>
>> >> Something like this should work on ubuntu.
>> >>
>> >> https://paste.apache.org/6kao?action=download
>> >>
>> >>
>> >> On Sun, Apr 6, 2014 at 11:11 PM, Check Peck <co...@gmail.com>
>> >> wrote:
>> >> > I am not sure from where to start to install zkpython.. I tried
>> >> > following
>> >> > that github link but was not able to understand. If you can guide me
>> >> > what I
>> >> > am supposed to do step by step, then it will be of great help to me.
>> >> > Thanks.
>> >> >
>> >> >
>> >> > On Sun, Apr 6, 2014 at 10:57 PM, Pavan Sudheendra
>> >> > <pa...@gmail.com>wrote:
>> >> >
>> >> >> Hi Check,
>> >> >>
>> >> >> Where are you getting stuck?
>> >> >>
>> >> >>
>> >> >> On Mon, Apr 7, 2014 at 11:13 AM, Check Peck
>> >> >> <co...@gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >> > Thanks. I tried going through that github link but wasn't able to
>> >> >> > understand what I am supposed to do clearly. May be I am not that
>> >> >> > much
>> >> >> > familiar with Python so that's why I am having hard time figuring
>> >> >> > that
>> >> >> out?
>> >> >> >
>> >> >> >
>> >> >> > On Sun, Apr 6, 2014 at 10:06 PM, Michi Mutsuzaki
>> >> >> > <michi@cs.stanford.edu
>> >> >> > >wrote:
>> >> >> >
>> >> >> > > You need to install zkpython. The steps are described in the
>> >> >> > > README
>> >> >> > > file. Let us know if you have trouble installing zkpython.
>> >> >> > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >> >>
>> >> >> https://github.com/apache/zookeeper/blob/trunk/src/contrib/zkpython/README
>> >> >> > >
>> >> >> > > On Sun, Apr 6, 2014 at 9:21 PM, Check Peck
>> >> >> > > <co...@gmail.com>
>> >> >> > > wrote:
>> >> >> > > > I tried following the steps mentioned on this github link -
>> >> >> > > > https://github.com/phunt/zk-smoketest for zk-latencies.py.
>> >> >> > > >
>> >> >> > > > I have 3 node ZK cluster- machineA, machineB, machineC
>> >> >> > > >
>> >> >> > > > Now I copied the Python program zk-latencies.py and
>> >> >> > > > zkclient.py
>> >> >> > > > in
>> >> >> > > another
>> >> >> > > > machineD in which no zookeeper is running. It's a plain ubuntu
>> >> >> > > > box.
>> >> >> > > >
>> >> >> > > > Now when I ran the program like this -
>> >> >> > > >
>> >> >> > > > david@machineD:~/zooksmoketest$ ./zk-latencies.py --servers
>> >> >> > > > "machineA:2181,machineB::2181,machineC:2181" --znode_count=100
>> >> >> > > > --znode_size=100 --synchronous
>> >> >> > > > Traceback (most recent call last):
>> >> >> > > > File "./zk-latencies.py", line 22, in <module>
>> >> >> > > > import zkclient
>> >> >> > > > File "/home/david/zooksmoketest/zkclient.py", line 17, in
>> >> >> > > > <module>
>> >> >> > > > import zookeeper, time, threading
>> >> >> > > >
>> >> >> > > > *ImportError: No module named zookeeper*
>> >> >> > > >
>> >> >> > > > I got an error as shown above. Any thoughts what wrong I am
>> >> >> > > > doing
>> >> >> > here? I
>> >> >> > > > don't have that much experience with Python. Any suggestions
>> >> >> > > > will
>> >> >> > > > be
>> >> >> of
>> >> >> > > > great help.
>> >> >> > > >
>> >> >> > > > @Raul: Any thoughts?
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > > On Sun, Apr 6, 2014 at 6:46 PM, Check Peck
>> >> >> > > > <co...@gmail.com>
>> >> >> > > wrote:
>> >> >> > > >
>> >> >> > > >> Thanks guys.. I am going to do load test in couple of days
>> >> >> > > >> using
>> >> >> > > >> the
>> >> >> > > >> exiting Python framework. And I will share the results once I
>> >> >> > > >> am
>> >> >> done.
>> >> >> > > >>
>> >> >> > > >>
>> >> >> > > >> On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R
>> >> >> > > >> <ra...@huawei.com>
>> >> >> > wrote:
>> >> >> > > >>
>> >> >> > > >>>
>> >> >> > > >>> If I understood the case correctly,
>> >> >> > > >>> Test :- 3 node ZK cluster, 600 sessions each will do one
>> >> >> > > >>> read
>> >> >> > > >>> operation(getChildren call) every one minute.
>> >> >> > > >>>
>> >> >> > > >>> I'm assuming the calls are equally distributed in time frame
>> >> >> > > >>> and it
>> >> >> > > would
>> >> >> > > >>> be like : 600ops/60secs, average 10 ops/sec will go to 3
>> >> >> > > >>> node
>> >> >> cluster
>> >> >> > > >>> I also feel it should be, also it would be great if you
>> >> >> > > >>> publish
>> >> >> > > >>> the
>> >> >> > > >>> results after your tests.
>> >> >> > > >>>
>> >> >> > > >>>
>> >> >> > > >>> -Rakesh
>> >> >> > > >>>
>> >> >> > > >>> -----Original Message-----
>> >> >> > > >>> From: Check Peck [mailto:comptechgeeky@gmail.com]
>> >> >> > > >>> Sent: 26 March 2014 11:19
>> >> >> > > >>> To: user
>> >> >> > > >>> Subject: Re: Load on Zookeeper cluster?
>> >> >> > > >>>
>> >> >> > > >>> Thanks that is also quite useful.
>> >> >> > > >>>
>> >> >> > > >>> For my above use case, I just need to get the children of a
>> >> >> > particular
>> >> >> > > >>> znode.. And that's what will be happening from all the 600
>> >> >> > application
>> >> >> > > >>> machines..
>> >> >> > > >>>
>> >> >> > > >>> From the wiki you provided, it looks it should be fine
>> >> >> > > >>> right?
>> >> >> > > >>>
>> >> >> > > >>>
>> >> >> > > >>> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R
>> >> >> > > >>> <ra...@huawei.com>
>> >> >> > wrote:
>> >> >> > > >>>
>> >> >> > > >>> >
>> >> >> > > >>> > Please go through the below one, hope it will be helpful
>> >> >> > > >>> > to
>> >> >> > > >>> > you.
>> >> >> > > Also,
>> >> >> > > >>> > I would recommend to have a look at the basic zk
>> >> >> > > >>> > configurations
>> >> >> > too.
>> >> >> > > >>> > All the
>> >> >> > > >>> > best:)
>> >> >> > > >>> >
>> >> >> > > >>> >
>> >> >> > > >>> >
>> >> >> > >
>> >> >> > >
>> >> >> > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
>> >> >> > > >>> > erview
>> >> >> > > >>> >
>> >> >> > > >>> >
>> >> >> > > >>> > What client operations(for ex: simply creating many
>> >> >> > > >>> > znodes,
>> >> >> > deleting,
>> >> >> > > >>> > setwatches et.) you are planning to perform.
>> >> >> > > >>> > Actually I'd like to know what each zkclient will be
>> >> >> > > >>> > doing.
>> >> >> > > >>> >
>> >> >> > > >>> > -Rakesh
>> >> >> > > >>> >
>> >> >> > > >>> > -----Original Message-----
>> >> >> > > >>> > From: Check Peck [mailto:comptechgeeky@gmail.com]
>> >> >> > > >>> > Sent: 26 March 2014 09:40
>> >> >> > > >>> > To: user
>> >> >> > > >>> > Subject: Re: Load on Zookeeper cluster?
>> >> >> > > >>> >
>> >> >> > > >>> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés <
>> >> >> > > >>> > rgs@itevenworks.net
>> >> >> > > >>> > > wrote:
>> >> >> > > >>> >
>> >> >> > > >>> > > 1 ZK se
>> >> >> > > >>> >
>> >> >> > > >>> >
>> >> >> > > >>> >
>> >> >> > > >>> > Yes one ZK session per application. Zookeeper version I am
>> >> >> running
>> >> >> > > >>> > currently is 3.4.5 in production.
>> >> >> > > >>> >
>> >> >> > > >>>
>> >> >> > > >>
>> >> >> > > >>
>> >> >> > >
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Regards-
>> >> >> Pavan
>> >> >>
>> >
>> >
>
>
Re: Load on Zookeeper cluster?
Posted by Check Peck <co...@gmail.com>.
I manually copied the jar file into the directory it was trying to copy.
Now if I do "ant", then I see like this -
check@phx5tg03c-cfc7:~/zooksmoketest/zookeeper-3.4.6$ ant
Buildfile: /home/check/zooksmoketest/zookeeper-3.4.6/build.xml
init:
ivy-download:
ivy-taskdef:
ivy-init:
ivy-retrieve:
[ivy:retrieve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/::
[ivy:retrieve] :: loading settings :: file =
/home/check/zooksmoketest/zookeeper-3.4.6/ivysettings.xml
[ivy:retrieve] :: resolving dependencies ::
org.apache.zookeeper#zookeeper;3.4.6
[ivy:retrieve] confs: [default]
And then after this, it doesn't move further, I waited for a long time and
nothing came up.. And afterwards I did ctrl + c to break.
Any idea what might be wrong?
On Mon, Apr 7, 2014 at 8:42 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>wrote:
> You probably need to set the proxy.
>
> http://ant.apache.org/manual/proxy.html
>
> On Mon, Apr 7, 2014 at 8:40 PM, Check Peck <co...@gmail.com>
> wrote:
> > Thanks a lot Michi. Appreciated your help.. I tried your suggestion on
> one
> > of my production box but when I tried "fifth" step which is "ant". I got
> an
> > error as shown below. Any thoughts what is wrong? I guess firewall is
> > blocking. Is there any other way around for this? I manually copied
> > "zookeeper-3.4.6.tar.gz" into that machine using "scp" as well.
> > check@phx5tg03c-cfc7:~/zooksmoketest/zookeeper-3.4.6$ ant
> > Buildfile: /home/check/zooksmoketest/zookeeper-3.4.6/build.xml
> >
> > init:
> >
> > ivy-download:
> > [get] Getting:
> > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
> > [get] To:
> > /home/check/zooksmoketest/zookeeper-3.4.6/src/java/lib/ivy-2.2.0.jar
> > [get] Error getting
> > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar to
> > /home/check/zooksmoketest/zookeeper-3.4.6/src/java/lib/ivy-2.2.0.jar
> >
> > BUILD FAILED
> > /home/check/zooksmoketest/zookeeper-3.4.6/build.xml:348:
> > java.net.ConnectException: Connection timed out
> > at java.net.PlainSocketImpl.socketConnect(Native Method)
> > at
> >
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
> > at
> >
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
> > at
> >
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
> > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
> > at java.net.Socket.connect(Socket.java:546)
> > at java.net.Socket.connect(Socket.java:495)
> > at sun.net.NetworkClient.doConnect(NetworkClient.java:178)
> > at sun.net.www.http.HttpClient.openServer(HttpClient.java:409)
> > at sun.net.www.http.HttpClient.openServer(HttpClient.java:530)
> > at sun.net.www.http.HttpClient.<init>(HttpClient.java:240)
> > at sun.net.www.http.HttpClient.New(HttpClient.java:321)
> > at sun.net.www.http.HttpClient.New(HttpClient.java:338)
> > at
> >
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:935)
> > at
> >
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:876)
> > at
> >
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:801)
> > at
> > org.apache.tools.ant.taskdefs.Get$GetThread.openConnection(Get.java:660)
> > at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:579)
> > at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569)
> >
> > Total time: 1 minute 3 seconds
> >
> >
> >
> > On Mon, Apr 7, 2014 at 3:08 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>
> > wrote:
> >>
> >> Hi Check,
> >>
> >> Something like this should work on ubuntu.
> >>
> >> https://paste.apache.org/6kao?action=download
> >>
> >>
> >> On Sun, Apr 6, 2014 at 11:11 PM, Check Peck <co...@gmail.com>
> >> wrote:
> >> > I am not sure from where to start to install zkpython.. I tried
> >> > following
> >> > that github link but was not able to understand. If you can guide me
> >> > what I
> >> > am supposed to do step by step, then it will be of great help to me.
> >> > Thanks.
> >> >
> >> >
> >> > On Sun, Apr 6, 2014 at 10:57 PM, Pavan Sudheendra
> >> > <pa...@gmail.com>wrote:
> >> >
> >> >> Hi Check,
> >> >>
> >> >> Where are you getting stuck?
> >> >>
> >> >>
> >> >> On Mon, Apr 7, 2014 at 11:13 AM, Check Peck <comptechgeeky@gmail.com
> >
> >> >> wrote:
> >> >>
> >> >> > Thanks. I tried going through that github link but wasn't able to
> >> >> > understand what I am supposed to do clearly. May be I am not that
> >> >> > much
> >> >> > familiar with Python so that's why I am having hard time figuring
> >> >> > that
> >> >> out?
> >> >> >
> >> >> >
> >> >> > On Sun, Apr 6, 2014 at 10:06 PM, Michi Mutsuzaki
> >> >> > <michi@cs.stanford.edu
> >> >> > >wrote:
> >> >> >
> >> >> > > You need to install zkpython. The steps are described in the
> README
> >> >> > > file. Let us know if you have trouble installing zkpython.
> >> >> > >
> >> >> > >
> >> >> >
> >> >>
> >> >>
> https://github.com/apache/zookeeper/blob/trunk/src/contrib/zkpython/README
> >> >> > >
> >> >> > > On Sun, Apr 6, 2014 at 9:21 PM, Check Peck
> >> >> > > <co...@gmail.com>
> >> >> > > wrote:
> >> >> > > > I tried following the steps mentioned on this github link -
> >> >> > > > https://github.com/phunt/zk-smoketest for zk-latencies.py.
> >> >> > > >
> >> >> > > > I have 3 node ZK cluster- machineA, machineB, machineC
> >> >> > > >
> >> >> > > > Now I copied the Python program zk-latencies.py and zkclient.py
> >> >> > > > in
> >> >> > > another
> >> >> > > > machineD in which no zookeeper is running. It's a plain ubuntu
> >> >> > > > box.
> >> >> > > >
> >> >> > > > Now when I ran the program like this -
> >> >> > > >
> >> >> > > > david@machineD:~/zooksmoketest$ ./zk-latencies.py --servers
> >> >> > > > "machineA:2181,machineB::2181,machineC:2181" --znode_count=100
> >> >> > > > --znode_size=100 --synchronous
> >> >> > > > Traceback (most recent call last):
> >> >> > > > File "./zk-latencies.py", line 22, in <module>
> >> >> > > > import zkclient
> >> >> > > > File "/home/david/zooksmoketest/zkclient.py", line 17, in
> >> >> > > > <module>
> >> >> > > > import zookeeper, time, threading
> >> >> > > >
> >> >> > > > *ImportError: No module named zookeeper*
> >> >> > > >
> >> >> > > > I got an error as shown above. Any thoughts what wrong I am
> doing
> >> >> > here? I
> >> >> > > > don't have that much experience with Python. Any suggestions
> will
> >> >> > > > be
> >> >> of
> >> >> > > > great help.
> >> >> > > >
> >> >> > > > @Raul: Any thoughts?
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > On Sun, Apr 6, 2014 at 6:46 PM, Check Peck
> >> >> > > > <co...@gmail.com>
> >> >> > > wrote:
> >> >> > > >
> >> >> > > >> Thanks guys.. I am going to do load test in couple of days
> using
> >> >> > > >> the
> >> >> > > >> exiting Python framework. And I will share the results once I
> am
> >> >> done.
> >> >> > > >>
> >> >> > > >>
> >> >> > > >> On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R <
> rakeshr@huawei.com>
> >> >> > wrote:
> >> >> > > >>
> >> >> > > >>>
> >> >> > > >>> If I understood the case correctly,
> >> >> > > >>> Test :- 3 node ZK cluster, 600 sessions each will do one read
> >> >> > > >>> operation(getChildren call) every one minute.
> >> >> > > >>>
> >> >> > > >>> I'm assuming the calls are equally distributed in time frame
> >> >> > > >>> and it
> >> >> > > would
> >> >> > > >>> be like : 600ops/60secs, average 10 ops/sec will go to 3 node
> >> >> cluster
> >> >> > > >>> I also feel it should be, also it would be great if you
> publish
> >> >> > > >>> the
> >> >> > > >>> results after your tests.
> >> >> > > >>>
> >> >> > > >>>
> >> >> > > >>> -Rakesh
> >> >> > > >>>
> >> >> > > >>> -----Original Message-----
> >> >> > > >>> From: Check Peck [mailto:comptechgeeky@gmail.com]
> >> >> > > >>> Sent: 26 March 2014 11:19
> >> >> > > >>> To: user
> >> >> > > >>> Subject: Re: Load on Zookeeper cluster?
> >> >> > > >>>
> >> >> > > >>> Thanks that is also quite useful.
> >> >> > > >>>
> >> >> > > >>> For my above use case, I just need to get the children of a
> >> >> > particular
> >> >> > > >>> znode.. And that's what will be happening from all the 600
> >> >> > application
> >> >> > > >>> machines..
> >> >> > > >>>
> >> >> > > >>> From the wiki you provided, it looks it should be fine right?
> >> >> > > >>>
> >> >> > > >>>
> >> >> > > >>> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R <
> rakeshr@huawei.com>
> >> >> > wrote:
> >> >> > > >>>
> >> >> > > >>> >
> >> >> > > >>> > Please go through the below one, hope it will be helpful to
> >> >> > > >>> > you.
> >> >> > > Also,
> >> >> > > >>> > I would recommend to have a look at the basic zk
> >> >> > > >>> > configurations
> >> >> > too.
> >> >> > > >>> > All the
> >> >> > > >>> > best:)
> >> >> > > >>> >
> >> >> > > >>> >
> >> >> > > >>> >
> >> >> > >
> >> >> > >
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
> >> >> > > >>> > erview
> >> >> > > >>> >
> >> >> > > >>> >
> >> >> > > >>> > What client operations(for ex: simply creating many znodes,
> >> >> > deleting,
> >> >> > > >>> > setwatches et.) you are planning to perform.
> >> >> > > >>> > Actually I'd like to know what each zkclient will be doing.
> >> >> > > >>> >
> >> >> > > >>> > -Rakesh
> >> >> > > >>> >
> >> >> > > >>> > -----Original Message-----
> >> >> > > >>> > From: Check Peck [mailto:comptechgeeky@gmail.com]
> >> >> > > >>> > Sent: 26 March 2014 09:40
> >> >> > > >>> > To: user
> >> >> > > >>> > Subject: Re: Load on Zookeeper cluster?
> >> >> > > >>> >
> >> >> > > >>> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés <
> >> >> > > >>> > rgs@itevenworks.net
> >> >> > > >>> > > wrote:
> >> >> > > >>> >
> >> >> > > >>> > > 1 ZK se
> >> >> > > >>> >
> >> >> > > >>> >
> >> >> > > >>> >
> >> >> > > >>> > Yes one ZK session per application. Zookeeper version I am
> >> >> running
> >> >> > > >>> > currently is 3.4.5 in production.
> >> >> > > >>> >
> >> >> > > >>>
> >> >> > > >>
> >> >> > > >>
> >> >> > >
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Regards-
> >> >> Pavan
> >> >>
> >
> >
>
Re: Load on Zookeeper cluster?
Posted by Michi Mutsuzaki <mi...@cs.stanford.edu>.
You probably need to set the proxy.
http://ant.apache.org/manual/proxy.html
On Mon, Apr 7, 2014 at 8:40 PM, Check Peck <co...@gmail.com> wrote:
> Thanks a lot Michi. Appreciated your help.. I tried your suggestion on one
> of my production box but when I tried "fifth" step which is "ant". I got an
> error as shown below. Any thoughts what is wrong? I guess firewall is
> blocking. Is there any other way around for this? I manually copied
> "zookeeper-3.4.6.tar.gz" into that machine using "scp" as well.
> check@phx5tg03c-cfc7:~/zooksmoketest/zookeeper-3.4.6$ ant
> Buildfile: /home/check/zooksmoketest/zookeeper-3.4.6/build.xml
>
> init:
>
> ivy-download:
> [get] Getting:
> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
> [get] To:
> /home/check/zooksmoketest/zookeeper-3.4.6/src/java/lib/ivy-2.2.0.jar
> [get] Error getting
> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar to
> /home/check/zooksmoketest/zookeeper-3.4.6/src/java/lib/ivy-2.2.0.jar
>
> BUILD FAILED
> /home/check/zooksmoketest/zookeeper-3.4.6/build.xml:348:
> java.net.ConnectException: Connection timed out
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
> at java.net.Socket.connect(Socket.java:546)
> at java.net.Socket.connect(Socket.java:495)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:178)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:409)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:530)
> at sun.net.www.http.HttpClient.<init>(HttpClient.java:240)
> at sun.net.www.http.HttpClient.New(HttpClient.java:321)
> at sun.net.www.http.HttpClient.New(HttpClient.java:338)
> at
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:935)
> at
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:876)
> at
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:801)
> at
> org.apache.tools.ant.taskdefs.Get$GetThread.openConnection(Get.java:660)
> at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:579)
> at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569)
>
> Total time: 1 minute 3 seconds
>
>
>
> On Mon, Apr 7, 2014 at 3:08 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>
> wrote:
>>
>> Hi Check,
>>
>> Something like this should work on ubuntu.
>>
>> https://paste.apache.org/6kao?action=download
>>
>>
>> On Sun, Apr 6, 2014 at 11:11 PM, Check Peck <co...@gmail.com>
>> wrote:
>> > I am not sure from where to start to install zkpython.. I tried
>> > following
>> > that github link but was not able to understand. If you can guide me
>> > what I
>> > am supposed to do step by step, then it will be of great help to me.
>> > Thanks.
>> >
>> >
>> > On Sun, Apr 6, 2014 at 10:57 PM, Pavan Sudheendra
>> > <pa...@gmail.com>wrote:
>> >
>> >> Hi Check,
>> >>
>> >> Where are you getting stuck?
>> >>
>> >>
>> >> On Mon, Apr 7, 2014 at 11:13 AM, Check Peck <co...@gmail.com>
>> >> wrote:
>> >>
>> >> > Thanks. I tried going through that github link but wasn't able to
>> >> > understand what I am supposed to do clearly. May be I am not that
>> >> > much
>> >> > familiar with Python so that's why I am having hard time figuring
>> >> > that
>> >> out?
>> >> >
>> >> >
>> >> > On Sun, Apr 6, 2014 at 10:06 PM, Michi Mutsuzaki
>> >> > <michi@cs.stanford.edu
>> >> > >wrote:
>> >> >
>> >> > > You need to install zkpython. The steps are described in the README
>> >> > > file. Let us know if you have trouble installing zkpython.
>> >> > >
>> >> > >
>> >> >
>> >>
>> >> https://github.com/apache/zookeeper/blob/trunk/src/contrib/zkpython/README
>> >> > >
>> >> > > On Sun, Apr 6, 2014 at 9:21 PM, Check Peck
>> >> > > <co...@gmail.com>
>> >> > > wrote:
>> >> > > > I tried following the steps mentioned on this github link -
>> >> > > > https://github.com/phunt/zk-smoketest for zk-latencies.py.
>> >> > > >
>> >> > > > I have 3 node ZK cluster- machineA, machineB, machineC
>> >> > > >
>> >> > > > Now I copied the Python program zk-latencies.py and zkclient.py
>> >> > > > in
>> >> > > another
>> >> > > > machineD in which no zookeeper is running. It's a plain ubuntu
>> >> > > > box.
>> >> > > >
>> >> > > > Now when I ran the program like this -
>> >> > > >
>> >> > > > david@machineD:~/zooksmoketest$ ./zk-latencies.py --servers
>> >> > > > "machineA:2181,machineB::2181,machineC:2181" --znode_count=100
>> >> > > > --znode_size=100 --synchronous
>> >> > > > Traceback (most recent call last):
>> >> > > > File "./zk-latencies.py", line 22, in <module>
>> >> > > > import zkclient
>> >> > > > File "/home/david/zooksmoketest/zkclient.py", line 17, in
>> >> > > > <module>
>> >> > > > import zookeeper, time, threading
>> >> > > >
>> >> > > > *ImportError: No module named zookeeper*
>> >> > > >
>> >> > > > I got an error as shown above. Any thoughts what wrong I am doing
>> >> > here? I
>> >> > > > don't have that much experience with Python. Any suggestions will
>> >> > > > be
>> >> of
>> >> > > > great help.
>> >> > > >
>> >> > > > @Raul: Any thoughts?
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > On Sun, Apr 6, 2014 at 6:46 PM, Check Peck
>> >> > > > <co...@gmail.com>
>> >> > > wrote:
>> >> > > >
>> >> > > >> Thanks guys.. I am going to do load test in couple of days using
>> >> > > >> the
>> >> > > >> exiting Python framework. And I will share the results once I am
>> >> done.
>> >> > > >>
>> >> > > >>
>> >> > > >> On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R <ra...@huawei.com>
>> >> > wrote:
>> >> > > >>
>> >> > > >>>
>> >> > > >>> If I understood the case correctly,
>> >> > > >>> Test :- 3 node ZK cluster, 600 sessions each will do one read
>> >> > > >>> operation(getChildren call) every one minute.
>> >> > > >>>
>> >> > > >>> I'm assuming the calls are equally distributed in time frame
>> >> > > >>> and it
>> >> > > would
>> >> > > >>> be like : 600ops/60secs, average 10 ops/sec will go to 3 node
>> >> cluster
>> >> > > >>> I also feel it should be, also it would be great if you publish
>> >> > > >>> the
>> >> > > >>> results after your tests.
>> >> > > >>>
>> >> > > >>>
>> >> > > >>> -Rakesh
>> >> > > >>>
>> >> > > >>> -----Original Message-----
>> >> > > >>> From: Check Peck [mailto:comptechgeeky@gmail.com]
>> >> > > >>> Sent: 26 March 2014 11:19
>> >> > > >>> To: user
>> >> > > >>> Subject: Re: Load on Zookeeper cluster?
>> >> > > >>>
>> >> > > >>> Thanks that is also quite useful.
>> >> > > >>>
>> >> > > >>> For my above use case, I just need to get the children of a
>> >> > particular
>> >> > > >>> znode.. And that's what will be happening from all the 600
>> >> > application
>> >> > > >>> machines..
>> >> > > >>>
>> >> > > >>> From the wiki you provided, it looks it should be fine right?
>> >> > > >>>
>> >> > > >>>
>> >> > > >>> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R <ra...@huawei.com>
>> >> > wrote:
>> >> > > >>>
>> >> > > >>> >
>> >> > > >>> > Please go through the below one, hope it will be helpful to
>> >> > > >>> > you.
>> >> > > Also,
>> >> > > >>> > I would recommend to have a look at the basic zk
>> >> > > >>> > configurations
>> >> > too.
>> >> > > >>> > All the
>> >> > > >>> > best:)
>> >> > > >>> >
>> >> > > >>> >
>> >> > > >>> >
>> >> > >
>> >> > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
>> >> > > >>> > erview
>> >> > > >>> >
>> >> > > >>> >
>> >> > > >>> > What client operations(for ex: simply creating many znodes,
>> >> > deleting,
>> >> > > >>> > setwatches et.) you are planning to perform.
>> >> > > >>> > Actually I'd like to know what each zkclient will be doing.
>> >> > > >>> >
>> >> > > >>> > -Rakesh
>> >> > > >>> >
>> >> > > >>> > -----Original Message-----
>> >> > > >>> > From: Check Peck [mailto:comptechgeeky@gmail.com]
>> >> > > >>> > Sent: 26 March 2014 09:40
>> >> > > >>> > To: user
>> >> > > >>> > Subject: Re: Load on Zookeeper cluster?
>> >> > > >>> >
>> >> > > >>> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés <
>> >> > > >>> > rgs@itevenworks.net
>> >> > > >>> > > wrote:
>> >> > > >>> >
>> >> > > >>> > > 1 ZK se
>> >> > > >>> >
>> >> > > >>> >
>> >> > > >>> >
>> >> > > >>> > Yes one ZK session per application. Zookeeper version I am
>> >> running
>> >> > > >>> > currently is 3.4.5 in production.
>> >> > > >>> >
>> >> > > >>>
>> >> > > >>
>> >> > > >>
>> >> > >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Regards-
>> >> Pavan
>> >>
>
>
Re: Load on Zookeeper cluster?
Posted by Check Peck <co...@gmail.com>.
Thanks a lot Michi. Appreciated your help.. I tried your suggestion on one
of my production box but when I tried "fifth" step which is "ant". I got an
error as shown below. Any thoughts what is wrong? I guess firewall is
blocking. Is there any other way around for this? I manually copied
"zookeeper-3.4.6.tar.gz" into that machine using "scp" as well.
check@phx5tg03c-cfc7:~/zooksmoketest/zookeeper-3.4.6$ ant
Buildfile: /home/check/zooksmoketest/zookeeper-3.4.6/build.xml
init:
ivy-download:
[get] Getting:
http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
[get] To:
/home/check/zooksmoketest/zookeeper-3.4.6/src/java/lib/ivy-2.2.0.jar
[get] Error getting
http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar to
/home/check/zooksmoketest/zookeeper-3.4.6/src/java/lib/ivy-2.2.0.jar
BUILD FAILED
/home/check/zooksmoketest/zookeeper-3.4.6/build.xml:348:
java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
at java.net.Socket.connect(Socket.java:546)
at java.net.Socket.connect(Socket.java:495)
at sun.net.NetworkClient.doConnect(NetworkClient.java:178)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:409)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:530)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:240)
at sun.net.www.http.HttpClient.New(HttpClient.java:321)
at sun.net.www.http.HttpClient.New(HttpClient.java:338)
at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:935)
at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:876)
at
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:801)
at
org.apache.tools.ant.taskdefs.Get$GetThread.openConnection(Get.java:660)
at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:579)
at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569)
Total time: 1 minute 3 seconds
On Mon, Apr 7, 2014 at 3:08 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>wrote:
> Hi Check,
>
> Something like this should work on ubuntu.
>
> https://paste.apache.org/6kao?action=download
>
>
> On Sun, Apr 6, 2014 at 11:11 PM, Check Peck <co...@gmail.com>
> wrote:
> > I am not sure from where to start to install zkpython.. I tried following
> > that github link but was not able to understand. If you can guide me
> what I
> > am supposed to do step by step, then it will be of great help to me.
> Thanks.
> >
> >
> > On Sun, Apr 6, 2014 at 10:57 PM, Pavan Sudheendra <pavan0591@gmail.com
> >wrote:
> >
> >> Hi Check,
> >>
> >> Where are you getting stuck?
> >>
> >>
> >> On Mon, Apr 7, 2014 at 11:13 AM, Check Peck <co...@gmail.com>
> >> wrote:
> >>
> >> > Thanks. I tried going through that github link but wasn't able to
> >> > understand what I am supposed to do clearly. May be I am not that much
> >> > familiar with Python so that's why I am having hard time figuring that
> >> out?
> >> >
> >> >
> >> > On Sun, Apr 6, 2014 at 10:06 PM, Michi Mutsuzaki <
> michi@cs.stanford.edu
> >> > >wrote:
> >> >
> >> > > You need to install zkpython. The steps are described in the README
> >> > > file. Let us know if you have trouble installing zkpython.
> >> > >
> >> > >
> >> >
> >>
> https://github.com/apache/zookeeper/blob/trunk/src/contrib/zkpython/README
> >> > >
> >> > > On Sun, Apr 6, 2014 at 9:21 PM, Check Peck <comptechgeeky@gmail.com
> >
> >> > > wrote:
> >> > > > I tried following the steps mentioned on this github link -
> >> > > > https://github.com/phunt/zk-smoketest for zk-latencies.py.
> >> > > >
> >> > > > I have 3 node ZK cluster- machineA, machineB, machineC
> >> > > >
> >> > > > Now I copied the Python program zk-latencies.py and zkclient.py in
> >> > > another
> >> > > > machineD in which no zookeeper is running. It's a plain ubuntu
> box.
> >> > > >
> >> > > > Now when I ran the program like this -
> >> > > >
> >> > > > david@machineD:~/zooksmoketest$ ./zk-latencies.py --servers
> >> > > > "machineA:2181,machineB::2181,machineC:2181" --znode_count=100
> >> > > > --znode_size=100 --synchronous
> >> > > > Traceback (most recent call last):
> >> > > > File "./zk-latencies.py", line 22, in <module>
> >> > > > import zkclient
> >> > > > File "/home/david/zooksmoketest/zkclient.py", line 17, in
> <module>
> >> > > > import zookeeper, time, threading
> >> > > >
> >> > > > *ImportError: No module named zookeeper*
> >> > > >
> >> > > > I got an error as shown above. Any thoughts what wrong I am doing
> >> > here? I
> >> > > > don't have that much experience with Python. Any suggestions will
> be
> >> of
> >> > > > great help.
> >> > > >
> >> > > > @Raul: Any thoughts?
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Sun, Apr 6, 2014 at 6:46 PM, Check Peck <
> comptechgeeky@gmail.com>
> >> > > wrote:
> >> > > >
> >> > > >> Thanks guys.. I am going to do load test in couple of days using
> the
> >> > > >> exiting Python framework. And I will share the results once I am
> >> done.
> >> > > >>
> >> > > >>
> >> > > >> On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R <ra...@huawei.com>
> >> > wrote:
> >> > > >>
> >> > > >>>
> >> > > >>> If I understood the case correctly,
> >> > > >>> Test :- 3 node ZK cluster, 600 sessions each will do one read
> >> > > >>> operation(getChildren call) every one minute.
> >> > > >>>
> >> > > >>> I'm assuming the calls are equally distributed in time frame
> and it
> >> > > would
> >> > > >>> be like : 600ops/60secs, average 10 ops/sec will go to 3 node
> >> cluster
> >> > > >>> I also feel it should be, also it would be great if you publish
> the
> >> > > >>> results after your tests.
> >> > > >>>
> >> > > >>>
> >> > > >>> -Rakesh
> >> > > >>>
> >> > > >>> -----Original Message-----
> >> > > >>> From: Check Peck [mailto:comptechgeeky@gmail.com]
> >> > > >>> Sent: 26 March 2014 11:19
> >> > > >>> To: user
> >> > > >>> Subject: Re: Load on Zookeeper cluster?
> >> > > >>>
> >> > > >>> Thanks that is also quite useful.
> >> > > >>>
> >> > > >>> For my above use case, I just need to get the children of a
> >> > particular
> >> > > >>> znode.. And that's what will be happening from all the 600
> >> > application
> >> > > >>> machines..
> >> > > >>>
> >> > > >>> From the wiki you provided, it looks it should be fine right?
> >> > > >>>
> >> > > >>>
> >> > > >>> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R <ra...@huawei.com>
> >> > wrote:
> >> > > >>>
> >> > > >>> >
> >> > > >>> > Please go through the below one, hope it will be helpful to
> you.
> >> > > Also,
> >> > > >>> > I would recommend to have a look at the basic zk
> configurations
> >> > too.
> >> > > >>> > All the
> >> > > >>> > best:)
> >> > > >>> >
> >> > > >>> >
> >> > > >>> >
> >> > >
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
> >> > > >>> > erview
> >> > > >>> >
> >> > > >>> >
> >> > > >>> > What client operations(for ex: simply creating many znodes,
> >> > deleting,
> >> > > >>> > setwatches et.) you are planning to perform.
> >> > > >>> > Actually I'd like to know what each zkclient will be doing.
> >> > > >>> >
> >> > > >>> > -Rakesh
> >> > > >>> >
> >> > > >>> > -----Original Message-----
> >> > > >>> > From: Check Peck [mailto:comptechgeeky@gmail.com]
> >> > > >>> > Sent: 26 March 2014 09:40
> >> > > >>> > To: user
> >> > > >>> > Subject: Re: Load on Zookeeper cluster?
> >> > > >>> >
> >> > > >>> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés <
> >> > > >>> > rgs@itevenworks.net
> >> > > >>> > > wrote:
> >> > > >>> >
> >> > > >>> > > 1 ZK se
> >> > > >>> >
> >> > > >>> >
> >> > > >>> >
> >> > > >>> > Yes one ZK session per application. Zookeeper version I am
> >> running
> >> > > >>> > currently is 3.4.5 in production.
> >> > > >>> >
> >> > > >>>
> >> > > >>
> >> > > >>
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Regards-
> >> Pavan
> >>
>
Re: Load on Zookeeper cluster?
Posted by Michi Mutsuzaki <mi...@cs.stanford.edu>.
Hi Check,
Something like this should work on ubuntu.
https://paste.apache.org/6kao?action=download
On Sun, Apr 6, 2014 at 11:11 PM, Check Peck <co...@gmail.com> wrote:
> I am not sure from where to start to install zkpython.. I tried following
> that github link but was not able to understand. If you can guide me what I
> am supposed to do step by step, then it will be of great help to me. Thanks.
>
>
> On Sun, Apr 6, 2014 at 10:57 PM, Pavan Sudheendra <pa...@gmail.com>wrote:
>
>> Hi Check,
>>
>> Where are you getting stuck?
>>
>>
>> On Mon, Apr 7, 2014 at 11:13 AM, Check Peck <co...@gmail.com>
>> wrote:
>>
>> > Thanks. I tried going through that github link but wasn't able to
>> > understand what I am supposed to do clearly. May be I am not that much
>> > familiar with Python so that's why I am having hard time figuring that
>> out?
>> >
>> >
>> > On Sun, Apr 6, 2014 at 10:06 PM, Michi Mutsuzaki <michi@cs.stanford.edu
>> > >wrote:
>> >
>> > > You need to install zkpython. The steps are described in the README
>> > > file. Let us know if you have trouble installing zkpython.
>> > >
>> > >
>> >
>> https://github.com/apache/zookeeper/blob/trunk/src/contrib/zkpython/README
>> > >
>> > > On Sun, Apr 6, 2014 at 9:21 PM, Check Peck <co...@gmail.com>
>> > > wrote:
>> > > > I tried following the steps mentioned on this github link -
>> > > > https://github.com/phunt/zk-smoketest for zk-latencies.py.
>> > > >
>> > > > I have 3 node ZK cluster- machineA, machineB, machineC
>> > > >
>> > > > Now I copied the Python program zk-latencies.py and zkclient.py in
>> > > another
>> > > > machineD in which no zookeeper is running. It's a plain ubuntu box.
>> > > >
>> > > > Now when I ran the program like this -
>> > > >
>> > > > david@machineD:~/zooksmoketest$ ./zk-latencies.py --servers
>> > > > "machineA:2181,machineB::2181,machineC:2181" --znode_count=100
>> > > > --znode_size=100 --synchronous
>> > > > Traceback (most recent call last):
>> > > > File "./zk-latencies.py", line 22, in <module>
>> > > > import zkclient
>> > > > File "/home/david/zooksmoketest/zkclient.py", line 17, in <module>
>> > > > import zookeeper, time, threading
>> > > >
>> > > > *ImportError: No module named zookeeper*
>> > > >
>> > > > I got an error as shown above. Any thoughts what wrong I am doing
>> > here? I
>> > > > don't have that much experience with Python. Any suggestions will be
>> of
>> > > > great help.
>> > > >
>> > > > @Raul: Any thoughts?
>> > > >
>> > > >
>> > > >
>> > > > On Sun, Apr 6, 2014 at 6:46 PM, Check Peck <co...@gmail.com>
>> > > wrote:
>> > > >
>> > > >> Thanks guys.. I am going to do load test in couple of days using the
>> > > >> exiting Python framework. And I will share the results once I am
>> done.
>> > > >>
>> > > >>
>> > > >> On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R <ra...@huawei.com>
>> > wrote:
>> > > >>
>> > > >>>
>> > > >>> If I understood the case correctly,
>> > > >>> Test :- 3 node ZK cluster, 600 sessions each will do one read
>> > > >>> operation(getChildren call) every one minute.
>> > > >>>
>> > > >>> I'm assuming the calls are equally distributed in time frame and it
>> > > would
>> > > >>> be like : 600ops/60secs, average 10 ops/sec will go to 3 node
>> cluster
>> > > >>> I also feel it should be, also it would be great if you publish the
>> > > >>> results after your tests.
>> > > >>>
>> > > >>>
>> > > >>> -Rakesh
>> > > >>>
>> > > >>> -----Original Message-----
>> > > >>> From: Check Peck [mailto:comptechgeeky@gmail.com]
>> > > >>> Sent: 26 March 2014 11:19
>> > > >>> To: user
>> > > >>> Subject: Re: Load on Zookeeper cluster?
>> > > >>>
>> > > >>> Thanks that is also quite useful.
>> > > >>>
>> > > >>> For my above use case, I just need to get the children of a
>> > particular
>> > > >>> znode.. And that's what will be happening from all the 600
>> > application
>> > > >>> machines..
>> > > >>>
>> > > >>> From the wiki you provided, it looks it should be fine right?
>> > > >>>
>> > > >>>
>> > > >>> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R <ra...@huawei.com>
>> > wrote:
>> > > >>>
>> > > >>> >
>> > > >>> > Please go through the below one, hope it will be helpful to you.
>> > > Also,
>> > > >>> > I would recommend to have a look at the basic zk configurations
>> > too.
>> > > >>> > All the
>> > > >>> > best:)
>> > > >>> >
>> > > >>> >
>> > > >>> >
>> > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
>> > > >>> > erview
>> > > >>> >
>> > > >>> >
>> > > >>> > What client operations(for ex: simply creating many znodes,
>> > deleting,
>> > > >>> > setwatches et.) you are planning to perform.
>> > > >>> > Actually I'd like to know what each zkclient will be doing.
>> > > >>> >
>> > > >>> > -Rakesh
>> > > >>> >
>> > > >>> > -----Original Message-----
>> > > >>> > From: Check Peck [mailto:comptechgeeky@gmail.com]
>> > > >>> > Sent: 26 March 2014 09:40
>> > > >>> > To: user
>> > > >>> > Subject: Re: Load on Zookeeper cluster?
>> > > >>> >
>> > > >>> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés <
>> > > >>> > rgs@itevenworks.net
>> > > >>> > > wrote:
>> > > >>> >
>> > > >>> > > 1 ZK se
>> > > >>> >
>> > > >>> >
>> > > >>> >
>> > > >>> > Yes one ZK session per application. Zookeeper version I am
>> running
>> > > >>> > currently is 3.4.5 in production.
>> > > >>> >
>> > > >>>
>> > > >>
>> > > >>
>> > >
>> >
>>
>>
>>
>> --
>> Regards-
>> Pavan
>>
Re: Load on Zookeeper cluster?
Posted by Check Peck <co...@gmail.com>.
I am not sure from where to start to install zkpython.. I tried following
that github link but was not able to understand. If you can guide me what I
am supposed to do step by step, then it will be of great help to me. Thanks.
On Sun, Apr 6, 2014 at 10:57 PM, Pavan Sudheendra <pa...@gmail.com>wrote:
> Hi Check,
>
> Where are you getting stuck?
>
>
> On Mon, Apr 7, 2014 at 11:13 AM, Check Peck <co...@gmail.com>
> wrote:
>
> > Thanks. I tried going through that github link but wasn't able to
> > understand what I am supposed to do clearly. May be I am not that much
> > familiar with Python so that's why I am having hard time figuring that
> out?
> >
> >
> > On Sun, Apr 6, 2014 at 10:06 PM, Michi Mutsuzaki <michi@cs.stanford.edu
> > >wrote:
> >
> > > You need to install zkpython. The steps are described in the README
> > > file. Let us know if you have trouble installing zkpython.
> > >
> > >
> >
> https://github.com/apache/zookeeper/blob/trunk/src/contrib/zkpython/README
> > >
> > > On Sun, Apr 6, 2014 at 9:21 PM, Check Peck <co...@gmail.com>
> > > wrote:
> > > > I tried following the steps mentioned on this github link -
> > > > https://github.com/phunt/zk-smoketest for zk-latencies.py.
> > > >
> > > > I have 3 node ZK cluster- machineA, machineB, machineC
> > > >
> > > > Now I copied the Python program zk-latencies.py and zkclient.py in
> > > another
> > > > machineD in which no zookeeper is running. It's a plain ubuntu box.
> > > >
> > > > Now when I ran the program like this -
> > > >
> > > > david@machineD:~/zooksmoketest$ ./zk-latencies.py --servers
> > > > "machineA:2181,machineB::2181,machineC:2181" --znode_count=100
> > > > --znode_size=100 --synchronous
> > > > Traceback (most recent call last):
> > > > File "./zk-latencies.py", line 22, in <module>
> > > > import zkclient
> > > > File "/home/david/zooksmoketest/zkclient.py", line 17, in <module>
> > > > import zookeeper, time, threading
> > > >
> > > > *ImportError: No module named zookeeper*
> > > >
> > > > I got an error as shown above. Any thoughts what wrong I am doing
> > here? I
> > > > don't have that much experience with Python. Any suggestions will be
> of
> > > > great help.
> > > >
> > > > @Raul: Any thoughts?
> > > >
> > > >
> > > >
> > > > On Sun, Apr 6, 2014 at 6:46 PM, Check Peck <co...@gmail.com>
> > > wrote:
> > > >
> > > >> Thanks guys.. I am going to do load test in couple of days using the
> > > >> exiting Python framework. And I will share the results once I am
> done.
> > > >>
> > > >>
> > > >> On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R <ra...@huawei.com>
> > wrote:
> > > >>
> > > >>>
> > > >>> If I understood the case correctly,
> > > >>> Test :- 3 node ZK cluster, 600 sessions each will do one read
> > > >>> operation(getChildren call) every one minute.
> > > >>>
> > > >>> I'm assuming the calls are equally distributed in time frame and it
> > > would
> > > >>> be like : 600ops/60secs, average 10 ops/sec will go to 3 node
> cluster
> > > >>> I also feel it should be, also it would be great if you publish the
> > > >>> results after your tests.
> > > >>>
> > > >>>
> > > >>> -Rakesh
> > > >>>
> > > >>> -----Original Message-----
> > > >>> From: Check Peck [mailto:comptechgeeky@gmail.com]
> > > >>> Sent: 26 March 2014 11:19
> > > >>> To: user
> > > >>> Subject: Re: Load on Zookeeper cluster?
> > > >>>
> > > >>> Thanks that is also quite useful.
> > > >>>
> > > >>> For my above use case, I just need to get the children of a
> > particular
> > > >>> znode.. And that's what will be happening from all the 600
> > application
> > > >>> machines..
> > > >>>
> > > >>> From the wiki you provided, it looks it should be fine right?
> > > >>>
> > > >>>
> > > >>> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R <ra...@huawei.com>
> > wrote:
> > > >>>
> > > >>> >
> > > >>> > Please go through the below one, hope it will be helpful to you.
> > > Also,
> > > >>> > I would recommend to have a look at the basic zk configurations
> > too.
> > > >>> > All the
> > > >>> > best:)
> > > >>> >
> > > >>> >
> > > >>> >
> > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
> > > >>> > erview
> > > >>> >
> > > >>> >
> > > >>> > What client operations(for ex: simply creating many znodes,
> > deleting,
> > > >>> > setwatches et.) you are planning to perform.
> > > >>> > Actually I'd like to know what each zkclient will be doing.
> > > >>> >
> > > >>> > -Rakesh
> > > >>> >
> > > >>> > -----Original Message-----
> > > >>> > From: Check Peck [mailto:comptechgeeky@gmail.com]
> > > >>> > Sent: 26 March 2014 09:40
> > > >>> > To: user
> > > >>> > Subject: Re: Load on Zookeeper cluster?
> > > >>> >
> > > >>> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés <
> > > >>> > rgs@itevenworks.net
> > > >>> > > wrote:
> > > >>> >
> > > >>> > > 1 ZK se
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > Yes one ZK session per application. Zookeeper version I am
> running
> > > >>> > currently is 3.4.5 in production.
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > >
> >
>
>
>
> --
> Regards-
> Pavan
>
Re: Load on Zookeeper cluster?
Posted by Pavan Sudheendra <pa...@gmail.com>.
Hi Check,
Where are you getting stuck?
On Mon, Apr 7, 2014 at 11:13 AM, Check Peck <co...@gmail.com> wrote:
> Thanks. I tried going through that github link but wasn't able to
> understand what I am supposed to do clearly. May be I am not that much
> familiar with Python so that's why I am having hard time figuring that out?
>
>
> On Sun, Apr 6, 2014 at 10:06 PM, Michi Mutsuzaki <michi@cs.stanford.edu
> >wrote:
>
> > You need to install zkpython. The steps are described in the README
> > file. Let us know if you have trouble installing zkpython.
> >
> >
> https://github.com/apache/zookeeper/blob/trunk/src/contrib/zkpython/README
> >
> > On Sun, Apr 6, 2014 at 9:21 PM, Check Peck <co...@gmail.com>
> > wrote:
> > > I tried following the steps mentioned on this github link -
> > > https://github.com/phunt/zk-smoketest for zk-latencies.py.
> > >
> > > I have 3 node ZK cluster- machineA, machineB, machineC
> > >
> > > Now I copied the Python program zk-latencies.py and zkclient.py in
> > another
> > > machineD in which no zookeeper is running. It's a plain ubuntu box.
> > >
> > > Now when I ran the program like this -
> > >
> > > david@machineD:~/zooksmoketest$ ./zk-latencies.py --servers
> > > "machineA:2181,machineB::2181,machineC:2181" --znode_count=100
> > > --znode_size=100 --synchronous
> > > Traceback (most recent call last):
> > > File "./zk-latencies.py", line 22, in <module>
> > > import zkclient
> > > File "/home/david/zooksmoketest/zkclient.py", line 17, in <module>
> > > import zookeeper, time, threading
> > >
> > > *ImportError: No module named zookeeper*
> > >
> > > I got an error as shown above. Any thoughts what wrong I am doing
> here? I
> > > don't have that much experience with Python. Any suggestions will be of
> > > great help.
> > >
> > > @Raul: Any thoughts?
> > >
> > >
> > >
> > > On Sun, Apr 6, 2014 at 6:46 PM, Check Peck <co...@gmail.com>
> > wrote:
> > >
> > >> Thanks guys.. I am going to do load test in couple of days using the
> > >> exiting Python framework. And I will share the results once I am done.
> > >>
> > >>
> > >> On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R <ra...@huawei.com>
> wrote:
> > >>
> > >>>
> > >>> If I understood the case correctly,
> > >>> Test :- 3 node ZK cluster, 600 sessions each will do one read
> > >>> operation(getChildren call) every one minute.
> > >>>
> > >>> I'm assuming the calls are equally distributed in time frame and it
> > would
> > >>> be like : 600ops/60secs, average 10 ops/sec will go to 3 node cluster
> > >>> I also feel it should be, also it would be great if you publish the
> > >>> results after your tests.
> > >>>
> > >>>
> > >>> -Rakesh
> > >>>
> > >>> -----Original Message-----
> > >>> From: Check Peck [mailto:comptechgeeky@gmail.com]
> > >>> Sent: 26 March 2014 11:19
> > >>> To: user
> > >>> Subject: Re: Load on Zookeeper cluster?
> > >>>
> > >>> Thanks that is also quite useful.
> > >>>
> > >>> For my above use case, I just need to get the children of a
> particular
> > >>> znode.. And that's what will be happening from all the 600
> application
> > >>> machines..
> > >>>
> > >>> From the wiki you provided, it looks it should be fine right?
> > >>>
> > >>>
> > >>> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R <ra...@huawei.com>
> wrote:
> > >>>
> > >>> >
> > >>> > Please go through the below one, hope it will be helpful to you.
> > Also,
> > >>> > I would recommend to have a look at the basic zk configurations
> too.
> > >>> > All the
> > >>> > best:)
> > >>> >
> > >>> >
> > >>> >
> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
> > >>> > erview
> > >>> >
> > >>> >
> > >>> > What client operations(for ex: simply creating many znodes,
> deleting,
> > >>> > setwatches et.) you are planning to perform.
> > >>> > Actually I'd like to know what each zkclient will be doing.
> > >>> >
> > >>> > -Rakesh
> > >>> >
> > >>> > -----Original Message-----
> > >>> > From: Check Peck [mailto:comptechgeeky@gmail.com]
> > >>> > Sent: 26 March 2014 09:40
> > >>> > To: user
> > >>> > Subject: Re: Load on Zookeeper cluster?
> > >>> >
> > >>> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés <
> > >>> > rgs@itevenworks.net
> > >>> > > wrote:
> > >>> >
> > >>> > > 1 ZK se
> > >>> >
> > >>> >
> > >>> >
> > >>> > Yes one ZK session per application. Zookeeper version I am running
> > >>> > currently is 3.4.5 in production.
> > >>> >
> > >>>
> > >>
> > >>
> >
>
--
Regards-
Pavan
Re: Load on Zookeeper cluster?
Posted by Check Peck <co...@gmail.com>.
Thanks. I tried going through that github link but wasn't able to
understand what I am supposed to do clearly. May be I am not that much
familiar with Python so that's why I am having hard time figuring that out?
On Sun, Apr 6, 2014 at 10:06 PM, Michi Mutsuzaki <mi...@cs.stanford.edu>wrote:
> You need to install zkpython. The steps are described in the README
> file. Let us know if you have trouble installing zkpython.
>
> https://github.com/apache/zookeeper/blob/trunk/src/contrib/zkpython/README
>
> On Sun, Apr 6, 2014 at 9:21 PM, Check Peck <co...@gmail.com>
> wrote:
> > I tried following the steps mentioned on this github link -
> > https://github.com/phunt/zk-smoketest for zk-latencies.py.
> >
> > I have 3 node ZK cluster- machineA, machineB, machineC
> >
> > Now I copied the Python program zk-latencies.py and zkclient.py in
> another
> > machineD in which no zookeeper is running. It's a plain ubuntu box.
> >
> > Now when I ran the program like this -
> >
> > david@machineD:~/zooksmoketest$ ./zk-latencies.py --servers
> > "machineA:2181,machineB::2181,machineC:2181" --znode_count=100
> > --znode_size=100 --synchronous
> > Traceback (most recent call last):
> > File "./zk-latencies.py", line 22, in <module>
> > import zkclient
> > File "/home/david/zooksmoketest/zkclient.py", line 17, in <module>
> > import zookeeper, time, threading
> >
> > *ImportError: No module named zookeeper*
> >
> > I got an error as shown above. Any thoughts what wrong I am doing here? I
> > don't have that much experience with Python. Any suggestions will be of
> > great help.
> >
> > @Raul: Any thoughts?
> >
> >
> >
> > On Sun, Apr 6, 2014 at 6:46 PM, Check Peck <co...@gmail.com>
> wrote:
> >
> >> Thanks guys.. I am going to do load test in couple of days using the
> >> exiting Python framework. And I will share the results once I am done.
> >>
> >>
> >> On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R <ra...@huawei.com> wrote:
> >>
> >>>
> >>> If I understood the case correctly,
> >>> Test :- 3 node ZK cluster, 600 sessions each will do one read
> >>> operation(getChildren call) every one minute.
> >>>
> >>> I'm assuming the calls are equally distributed in time frame and it
> would
> >>> be like : 600ops/60secs, average 10 ops/sec will go to 3 node cluster
> >>> I also feel it should be, also it would be great if you publish the
> >>> results after your tests.
> >>>
> >>>
> >>> -Rakesh
> >>>
> >>> -----Original Message-----
> >>> From: Check Peck [mailto:comptechgeeky@gmail.com]
> >>> Sent: 26 March 2014 11:19
> >>> To: user
> >>> Subject: Re: Load on Zookeeper cluster?
> >>>
> >>> Thanks that is also quite useful.
> >>>
> >>> For my above use case, I just need to get the children of a particular
> >>> znode.. And that's what will be happening from all the 600 application
> >>> machines..
> >>>
> >>> From the wiki you provided, it looks it should be fine right?
> >>>
> >>>
> >>> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R <ra...@huawei.com> wrote:
> >>>
> >>> >
> >>> > Please go through the below one, hope it will be helpful to you.
> Also,
> >>> > I would recommend to have a look at the basic zk configurations too.
> >>> > All the
> >>> > best:)
> >>> >
> >>> >
> >>> >
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
> >>> > erview
> >>> >
> >>> >
> >>> > What client operations(for ex: simply creating many znodes, deleting,
> >>> > setwatches et.) you are planning to perform.
> >>> > Actually I'd like to know what each zkclient will be doing.
> >>> >
> >>> > -Rakesh
> >>> >
> >>> > -----Original Message-----
> >>> > From: Check Peck [mailto:comptechgeeky@gmail.com]
> >>> > Sent: 26 March 2014 09:40
> >>> > To: user
> >>> > Subject: Re: Load on Zookeeper cluster?
> >>> >
> >>> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés <
> >>> > rgs@itevenworks.net
> >>> > > wrote:
> >>> >
> >>> > > 1 ZK se
> >>> >
> >>> >
> >>> >
> >>> > Yes one ZK session per application. Zookeeper version I am running
> >>> > currently is 3.4.5 in production.
> >>> >
> >>>
> >>
> >>
>
Re: Load on Zookeeper cluster?
Posted by Michi Mutsuzaki <mi...@cs.stanford.edu>.
You need to install zkpython. The steps are described in the README
file. Let us know if you have trouble installing zkpython.
https://github.com/apache/zookeeper/blob/trunk/src/contrib/zkpython/README
On Sun, Apr 6, 2014 at 9:21 PM, Check Peck <co...@gmail.com> wrote:
> I tried following the steps mentioned on this github link -
> https://github.com/phunt/zk-smoketest for zk-latencies.py.
>
> I have 3 node ZK cluster- machineA, machineB, machineC
>
> Now I copied the Python program zk-latencies.py and zkclient.py in another
> machineD in which no zookeeper is running. It's a plain ubuntu box.
>
> Now when I ran the program like this -
>
> david@machineD:~/zooksmoketest$ ./zk-latencies.py --servers
> "machineA:2181,machineB::2181,machineC:2181" --znode_count=100
> --znode_size=100 --synchronous
> Traceback (most recent call last):
> File "./zk-latencies.py", line 22, in <module>
> import zkclient
> File "/home/david/zooksmoketest/zkclient.py", line 17, in <module>
> import zookeeper, time, threading
>
> *ImportError: No module named zookeeper*
>
> I got an error as shown above. Any thoughts what wrong I am doing here? I
> don't have that much experience with Python. Any suggestions will be of
> great help.
>
> @Raul: Any thoughts?
>
>
>
> On Sun, Apr 6, 2014 at 6:46 PM, Check Peck <co...@gmail.com> wrote:
>
>> Thanks guys.. I am going to do load test in couple of days using the
>> exiting Python framework. And I will share the results once I am done.
>>
>>
>> On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R <ra...@huawei.com> wrote:
>>
>>>
>>> If I understood the case correctly,
>>> Test :- 3 node ZK cluster, 600 sessions each will do one read
>>> operation(getChildren call) every one minute.
>>>
>>> I'm assuming the calls are equally distributed in time frame and it would
>>> be like : 600ops/60secs, average 10 ops/sec will go to 3 node cluster
>>> I also feel it should be, also it would be great if you publish the
>>> results after your tests.
>>>
>>>
>>> -Rakesh
>>>
>>> -----Original Message-----
>>> From: Check Peck [mailto:comptechgeeky@gmail.com]
>>> Sent: 26 March 2014 11:19
>>> To: user
>>> Subject: Re: Load on Zookeeper cluster?
>>>
>>> Thanks that is also quite useful.
>>>
>>> For my above use case, I just need to get the children of a particular
>>> znode.. And that's what will be happening from all the 600 application
>>> machines..
>>>
>>> From the wiki you provided, it looks it should be fine right?
>>>
>>>
>>> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R <ra...@huawei.com> wrote:
>>>
>>> >
>>> > Please go through the below one, hope it will be helpful to you. Also,
>>> > I would recommend to have a look at the basic zk configurations too.
>>> > All the
>>> > best:)
>>> >
>>> >
>>> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
>>> > erview
>>> >
>>> >
>>> > What client operations(for ex: simply creating many znodes, deleting,
>>> > setwatches et.) you are planning to perform.
>>> > Actually I'd like to know what each zkclient will be doing.
>>> >
>>> > -Rakesh
>>> >
>>> > -----Original Message-----
>>> > From: Check Peck [mailto:comptechgeeky@gmail.com]
>>> > Sent: 26 March 2014 09:40
>>> > To: user
>>> > Subject: Re: Load on Zookeeper cluster?
>>> >
>>> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés <
>>> > rgs@itevenworks.net
>>> > > wrote:
>>> >
>>> > > 1 ZK se
>>> >
>>> >
>>> >
>>> > Yes one ZK session per application. Zookeeper version I am running
>>> > currently is 3.4.5 in production.
>>> >
>>>
>>
>>
Re: Load on Zookeeper cluster?
Posted by Check Peck <co...@gmail.com>.
I tried following the steps mentioned on this github link -
https://github.com/phunt/zk-smoketest for zk-latencies.py.
I have 3 node ZK cluster- machineA, machineB, machineC
Now I copied the Python program zk-latencies.py and zkclient.py in another
machineD in which no zookeeper is running. It's a plain ubuntu box.
Now when I ran the program like this -
david@machineD:~/zooksmoketest$ ./zk-latencies.py --servers
"machineA:2181,machineB::2181,machineC:2181" --znode_count=100
--znode_size=100 --synchronous
Traceback (most recent call last):
File "./zk-latencies.py", line 22, in <module>
import zkclient
File "/home/david/zooksmoketest/zkclient.py", line 17, in <module>
import zookeeper, time, threading
*ImportError: No module named zookeeper*
I got an error as shown above. Any thoughts what wrong I am doing here? I
don't have that much experience with Python. Any suggestions will be of
great help.
@Raul: Any thoughts?
On Sun, Apr 6, 2014 at 6:46 PM, Check Peck <co...@gmail.com> wrote:
> Thanks guys.. I am going to do load test in couple of days using the
> exiting Python framework. And I will share the results once I am done.
>
>
> On Tue, Mar 25, 2014 at 11:19 PM, Rakesh R <ra...@huawei.com> wrote:
>
>>
>> If I understood the case correctly,
>> Test :- 3 node ZK cluster, 600 sessions each will do one read
>> operation(getChildren call) every one minute.
>>
>> I'm assuming the calls are equally distributed in time frame and it would
>> be like : 600ops/60secs, average 10 ops/sec will go to 3 node cluster
>> I also feel it should be, also it would be great if you publish the
>> results after your tests.
>>
>>
>> -Rakesh
>>
>> -----Original Message-----
>> From: Check Peck [mailto:comptechgeeky@gmail.com]
>> Sent: 26 March 2014 11:19
>> To: user
>> Subject: Re: Load on Zookeeper cluster?
>>
>> Thanks that is also quite useful.
>>
>> For my above use case, I just need to get the children of a particular
>> znode.. And that's what will be happening from all the 600 application
>> machines..
>>
>> From the wiki you provided, it looks it should be fine right?
>>
>>
>> On Tue, Mar 25, 2014 at 10:13 PM, Rakesh R <ra...@huawei.com> wrote:
>>
>> >
>> > Please go through the below one, hope it will be helpful to you. Also,
>> > I would recommend to have a look at the basic zk configurations too.
>> > All the
>> > best:)
>> >
>> >
>> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOv
>> > erview
>> >
>> >
>> > What client operations(for ex: simply creating many znodes, deleting,
>> > setwatches et.) you are planning to perform.
>> > Actually I'd like to know what each zkclient will be doing.
>> >
>> > -Rakesh
>> >
>> > -----Original Message-----
>> > From: Check Peck [mailto:comptechgeeky@gmail.com]
>> > Sent: 26 March 2014 09:40
>> > To: user
>> > Subject: Re: Load on Zookeeper cluster?
>> >
>> > On Tue, Mar 25, 2014 at 9:00 PM, Raúl Gutiérrez Segalés <
>> > rgs@itevenworks.net
>> > > wrote:
>> >
>> > > 1 ZK se
>> >
>> >
>> >
>> > Yes one ZK session per application. Zookeeper version I am running
>> > currently is 3.4.5 in production.
>> >
>>
>
>