You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hbase.apache.org by Dalia Sobhy <da...@hotmail.com> on 2013/02/01 13:11:03 UTC

Hbase Read/Write throughput measure

Dear all,

I want to measure the read/write throughput for a code on a cluster of 10 nodes. So is there any code or way to measure it?

I have seen in a cloudera-based presentation that hbase read/write throughput = millions queries per second.

So any help please??

Thanks


Best Regards,
Dalia Sobhy

Re: Hbase Read/Write throughput measure

Posted by Mohammad Tariq <do...@gmail.com>.

ycsb-1.0.4 was built with with hbase-0.90.5 and you are usin 0.92, so you
need to update ycsb's hbase zookeeper jars. To do that copy the hbase-*.jar
and zookeeper-*.jar into the /hadoop/projects/ycsb-0.1.4/hbase-binding/lib
directory. And copy your hbase-site.xml
into /hadoop/projects/ycsb-0.1.4/hbase-binding/conf directory.

HTH

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Mon, Feb 4, 2013 at 12:17 AM, Mohammad Tariq <do...@gmail.com> wrote:

> Hello Dalia,
>
>           Me and Anil are talking about this :
>
> 1 - git clone http://github.com/brianfrankcooper/YCSB.git
> 2 - cd YCSB/
> 3 - mvn compile
>
> And not compiling through Java
>
> Warm Regards,
> Tariq
>  https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Mon, Feb 4, 2013 at 12:14 AM, Dalia Sobhy <da...@hotmail.com>wrote:
>
>> Hi anil,
>>
>> How to recompile it ?
>>
>> Sent from my iPhone
>>
>> On 2013-02-03, at 8:00 PM, "anil gupta" <an...@gmail.com> wrote:
>>
>> > Hi Dalia,
>> >
>> > Recently, i ran into same problem with YCSB. You will need to recompile
>> > YCSB with HBase0.92. Download the maven project from yscb site and
>> > recompile it with HBase0.92.
>> >
>> > HTH,
>> > Anil Gupta
>> >
>> > On Sun, Feb 3, 2013 at 9:49 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
>> >wrote:
>> >
>> >>
>> >> Hello Mohamed,
>> >>
>> >> Yes Hbase is running good and already I have a table and and some
>> records
>> >> there.
>> >>
>> >> I have done what you told me I copied all jar files from lib directory
>> to
>> >> YSCB/db/hbase/lib/ directory. After restart and writing this command
>> >> "./bin/ycsb run hbase -p columnfamily=family -P workloads/workloada", I
>> >> receive the same error.
>> >>
>> >> But I didn't compile, because already I don't have a .java file
>> >>
>> >> ycsb-0.1.4 directory doesn't contain hbase client java file, it
>> contains
>> >> only this "hbase-binding-0.1.4.jar"
>> >>
>> >> So what to do ?
>> >>
>> >>
>> >>> From: dontariq@gmail.com
>> >>> Date: Sun, 3 Feb 2013 22:54:13 +0530
>> >>> Subject: Re: Hbase Read/Write throughput measure
>> >>> To: user@hbase.apache.org
>> >>>
>> >>> Hello Dalia,
>> >>>
>> >>>        Is your Hbase running OK? Copy the hbase jar from your
>> HBASE_HOME
>> >>> and all the jars from your HBASE_HOME/lib/ directory to your
>> >>> YSCB/db/hbase/lib/ directory, compile it again and restart everything.
>> >>>
>> >>> Warm Regards,
>> >>> Tariq
>> >>> https://mtariq.jux.com/
>> >>> cloudfront.blogspot.com
>> >>>
>> >>>
>> >>> On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <
>> dalia.mohsobhy@hotmail.com
>> >>> wrote:
>> >>>
>> >>>>
>> >>>> Hello Mohamed,
>> >>>>
>> >>>> I believe thats not the case.
>> >>>>
>> >>>> check this blog
>> >>
>> http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
>> >>>>
>> >>>> It denotes that this error is due to incompatible Hbase client and
>> >> server
>> >>>> versions, although I am using Hbase server :0.92 and the server is
>> the
>> >>>> latest one uploaded by ycsb. So I don't know why this is happening.
>> >>>>
>> >>>> So do you have any idea?
>> >>>>
>> >>>> This is the cmd I am executing:
>> >>>>
>> >>>> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p
>> >> columnfamily="family"
>> >>>> -P workloads/workloada
>> >>>>
>> >>>> java -cp
>> >>
>> /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
>> >>>> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
>> >>>> columnfamily=family -P workloads/workloada -t
>> >>>> YCSB Client 0.1
>> >>>> Command line: -db com.yahoo.ycsb.db.HBaseClient -p
>> columnfamily=family
>> >> -P
>> >>>> workloads/workloada -t
>> >>>> Loading workload...
>> >>>> Starting test.
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>>> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010
>> 05:32
>> >> GMT
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
>> >> host.name=
>> >>>> master1.medcloud.com
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>>> environment:java.version=1.6.0_31
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>>> environment:java.vendor=Sun Microsystems Inc.
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>>> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>
>> environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>
>> environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>>> environment:java.io.tmpdir=/tmp
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>>> environment:java.compiler=<NA>
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
>> os.name
>> >>>> =Linux
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>>> environment:os.arch=amd64
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>>> environment:os.version=3.2.0-23-generic
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
>> >> user.name
>> >>>> =root
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>>> environment:user.home=/root
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> >>>> environment:user.dir=/home/dalia/ycsb-0.1.4
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client
>> >> connection,
>> >>>> connectString=192.168.2.3:2181 sessionTimeout=180000
>> >> watcher=hconnection
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket
>> connection
>> >> to
>> >>>> server /192.168.2.3:2181
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection
>> >> established
>> >>>> to master1.medcloud.com/192.168.2.3:2181, initiating session
>> >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
>> >>>> complete on server master1.medcloud.com/192.168.2.3:2181, sessionid
>> =
>> >>>> 0x13c9b45ac70000a, negotiated timeout = 60000
>> >>>> java.lang.IllegalArgumentException: Not a host:port pair: �
>> >>>> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
>> >>>>
>> >>>>
>> >>>> my hbase-site.xml:
>> >>>>
>> >>>> <configuration>
>> >>>>  <property>
>> >>>>    <name>hbase.rootdir</name>
>> >>>>   <value>hdfs://192.168.2.3:20001/hbase</value>
>> >>>>
>> >>>> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
>> >>>>    <description>The directory shared by region servers.
>> >>>>    </description>
>> >>>>  </property>
>> >>>>  <property>
>> >>>>    <name>hbase.master</name>
>> >>>>        <value>192.168.2.3:60000</value>
>> >>>>    <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
>> >>>>    <description>The host and port that the HBase master runs at.
>> >>>>    </description>
>> >>>>  </property>
>> >>>>  <property>
>> >>>>    <name>hbase.zookeeper.quorum</name>
>> >>>>    <value>192.168.2.3</value>
>> >>>>    <!--<value><HBASE_MASTER_SERVER></value>-->
>> >>>>  </property>
>> >>>> </configuration>
>> >>>>
>> >>>> So any help please....
>> >>>>
>> >>>> Thanks,
>> >>>>
>> >>>>> From: dontariq@gmail.com
>> >>>>> Date: Sun, 3 Feb 2013 13:54:28 +0530
>> >>>>> Subject: Re: Hbase Read/Write throughput measure
>> >>>>> To: user@hbase.apache.org
>> >>>>>
>> >>>>> Hello Dalia,
>> >>>>>
>> >>>>>       I am sorry for late response, courtesy hectic schedule. Add
>> >> the
>> >>>>> hbase-site.xml file to the classpath and see if it works for you.
>> >>>>>
>> >>>>> Warm Regards,
>> >>>>> Tariq
>> >>>>> https://mtariq.jux.com/
>> >>>>> cloudfront.blogspot.com
>> >>>>>
>> >>>>>
>> >>>>> On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <
>> >> dalia.mohsobhy@hotmail.com
>> >>>>> wrote:
>> >>>>>
>> >>>>>>
>> >>>>>> I tried yahoo bench mark I got the following error:
>> >>>>>>
>> >>>>>> java.lang.IllegalArgumentException: Not a host:port
>> >>>>>>
>> >>>>>> So any help,
>> >>>>>>
>> >>>>>> Note that I am using hbase 0.92
>> >>>>>>
>> >>>>>> I have downloaded the lastest ycsb from github
>> >>>>>>
>> >>>>>>> From: dalia.mohsobhy@hotmail.com
>> >>>>>>> To: user@hbase.apache.org
>> >>>>>>> Subject: RE: Hbase Read/Write throughput measure
>> >>>>>>> Date: Sat, 2 Feb 2013 16:47:43 +0200
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> any help ??!!!!!!
>> >>>>>>>
>> >>>>>>>> Date: Fri, 1 Feb 2013 17:15:45 -0500
>> >>>>>>>> Subject: Re: Hbase Read/Write throughput measure
>> >>>>>>>> From: jean-marc@spaggiari.org
>> >>>>>>>> To: user@hbase.apache.org
>> >>>>>>>>
>> >>>>>>>> When you will have figure how the command is working, you will
>> >>>> have to
>> >>>>>>>> understand the output ;)
>> >>>>>>>>
>> >>>>>>>> This is how I just tried it from my HBase directory:
>> >>>>>>>> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
>> >> randomRead
>> >>>> 10
>> >>>>>>>>
>> >>>>>>>> And this is the output (last lines):
>> >>>>>>>> 10066326    56314
>> >>>>>>>> 10171183    56385
>> >>>>>>>> 10276040    56579
>> >>>>>>>> 10380897    56553
>> >>>>>>>>
>> >>>>>>>> Does ask me what it means ;) I have, so far, no idea.
>> >>>>>>>>
>> >>>>>>>> JM
>> >>>>>>>>
>> >>>>>>>> 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
>> >>>>>>>>> Dear Mohamed,
>> >>>>>>>>>
>> >>>>>>>>> I checked this link:
>> >>>>>>>>> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
>> >>>>>>>>>
>> >>>>>>>>> But how to use the commands there is no clue, note that I am
>> >>>> using
>> >>>>>> cloudera
>> >>>>>>>>> manager.
>> >>>>>>>>>
>> >>>>>>>>> The other yahoo link I couldn't understand how to use it.
>> >>>>>>>>>
>> >>>>>>>>> So any help please??
>> >>>>>>>>>
>> >>>>>>>>> Sent from my iPad
>> >>>>>>>>>
>> >>>>>>>>> On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <
>> >> dontariq@gmail.com
>> >>>>>
>> >>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Hello Dalia,
>> >>>>>>>>>>
>> >>>>>>>>>>         I think the easiest way to measure the read/write
>> >>>>>> throughput is
>> >>>>>>>>>> to use "PerformanceEvaluation" tool that comes with the
>> >> Hbase
>> >>>>>>>>>> distribution. It spawns a map-reduce job to do the
>> >> reads/writes
>> >>>> in
>> >>>>>>>>>> parallel. Apart from this there are several other ways to
>> >>>> benchmark
>> >>>>>> your
>> >>>>>>>>>> Hbase cluster like YSCB <
>> >>>> https://github.com/brianfrankcooper/YCSB>.
>> >>>>>> You
>> >>>>>>>>>> might find this
>> >>>>>>>>>> link<
>> >> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
>> >>>>>> useful
>> >>>>>>>>>> that talks about Hbase performance.
>> >>>>>>>>>>
>> >>>>>>>>>> HTH
>> >>>>>>>>>>
>> >>>>>>>>>> Warm Regards,
>> >>>>>>>>>> Tariq
>> >>>>>>>>>> https://mtariq.jux.com/
>> >>>>>>>>>> cloudfront.blogspot.com
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
>> >>>>>>>>>> <da...@hotmail.com>wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> Dear all,
>> >>>>>>>>>>>
>> >>>>>>>>>>> I want to measure the read/write throughput for a code on a
>> >>>>>> cluster of
>> >>>>>>>>>>> 10
>> >>>>>>>>>>> nodes. So is there any code or way to measure it?
>> >>>>>>>>>>>
>> >>>>>>>>>>> I have seen in a cloudera-based presentation that hbase
>> >>>> read/write
>> >>>>>>>>>>> throughput = millions queries per second.
>> >>>>>>>>>>>
>> >>>>>>>>>>> So any help please??
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> Best Regards,
>> >>>>>>>>>>> Dalia Sobhy
>> >
>> >
>> >
>> > --
>> > Thanks & Regards,
>> > Anil Gupta
>>
>
>

RE: Hbase Read/Write throughput measure

Posted by Dalia Sobhy <da...@hotmail.com>.

Dear Mohamed,

It finished compiling and I have changed the hbase-site.xml of ycsb to the one from cloudera.

I am following the steps in your blog, but first I couldn't find the pom.xml in YCSB/hbase directory. I found it only in YCSB general directory, here is the contents:

 <!-- Properties Management -->
  <properties>
    <maven.assembly.version>2.2.1</maven.assembly.version>
    <hbase.version>0.92.1</hbase.version>
    <cassandra.version>0.7.0</cassandra.version>
    <infinispan.version>7.1.0.CR1</infinispan.version>
    <openjpa.jdbc.version>2.1.1</openjpa.jdbc.version>
    <mapkeeper.version>1.0</mapkeeper.version>                                          
    <mongodb.version>2.9.0</mongodb.version>
    <orientdb.version>1.0.1</orientdb.version>
    <redis.version>2.0.0</redis.version>
    <voldemort.version>0.81</voldemort.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <thrift.version>0.8.0</thrift.version>
    <hypertable.version>0.9.5.6</hypertable.version>
  </properties>

.....
....

After that I run the following from YCSB directory: mvn clean package
But here are he following errors:
/home/dalia/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:512:13: '{' should be on the previous line.
/home/dalia/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:512:13: Must have at least one statement.
/home/dalia/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:513: catch rcurly at indentation level 12 not at correct indentation, 6
/home/dalia/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:514: for rcurly at indentation level 8 not at correct indentation, 4
/home/dalia/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:515: method def child at indentation level 8 not at correct indentation, 4
/home/dalia/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:517: Line is longer than 80 characters.
/home/dalia/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:517: method def child at indentation level 8 not at correct indentation, 4
/home/dalia/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:519: method def rcurly at indentation level 4 not at correct indentation, 2
Audit done.

[INFO] There are 436 checkstyle errors.
[WARNING] Unable to locate Source XRef to link to - DISABLED
[INFO] 
[INFO] --- maven-resources-plugin:2.3:resources (default-resources) @ hbase-binding ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/dalia/YCSB/hbase/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ hbase-binding ---
[INFO] Compiling 1 source file to /home/dalia/YCSB/hbase/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.3:testResources (default-testResources) @ hbase-binding ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/dalia/YCSB/hbase/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ hbase-binding ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.10:test (default-test) @ hbase-binding ---
[INFO] No tests to run.
[INFO] Surefire report directory: /home/dalia/YCSB/hbase/target/surefire-reports

These errors are repeated for all the available databases ..

So any idea ?? 
> From: dontariq@gmail.com
> Date: Mon, 4 Feb 2013 02:48:25 +0530
> Subject: Re: Hbase Read/Write throughput measure
> To: user@hbase.apache.org
> 
> it may take some time depending on your connection. wait for sometime. if
> you still find some problem you can visit this
> link<http://cloudfront.blogspot.in/2013/02/how-to-benchmark-hbase-using-ycsb.html#.UQ7Tu5vS8s0>
> 
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
> 
> 
> On Mon, Feb 4, 2013 at 2:35 AM, Dalia Sobhy <da...@hotmail.com>wrote:
> 
> >
> > Hello Mohamed,
> >
> > I am using ubuntu 12, but the problem is that I was facing some errors
> > while compiling, after solving them.
> >
> > After $git clone http://github.com/brianfrankcooper/YCSB.git
> > Cloning into 'YCSB'...
> > remote: Counting objects: 2260, done.
> > remote: Compressing objects: 100% (694/694), done.
> > Receiving objects:  86% (1952/2260), 764.00 KiB | 51 KiB/s
> >
> > It keeps hanging on 86%, so any clue??
> >
> > Thanksss :D
> >
> > > From: dontariq@gmail.com
> > > Date: Mon, 4 Feb 2013 00:17:40 +0530
> > > Subject: Re: Hbase Read/Write throughput measure
> > > To: user@hbase.apache.org
> > >
> > > Hello Dalia,
> > >
> > >           Me and Anil are talking about this :
> > >
> > > 1 - git clone http://github.com/brianfrankcooper/YCSB.git
> > > 2 - cd YCSB/
> > > 3 - mvn compile
> > >
> > > And not compiling through Java
> > >
> > > Warm Regards,
> > > Tariq
> > > https://mtariq.jux.com/
> > > cloudfront.blogspot.com
> > >
> > >
> > > On Mon, Feb 4, 2013 at 12:14 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
> > >wrote:
> > >
> > > > Hi anil,
> > > >
> > > > How to recompile it ?
> > > >
> > > > Sent from my iPhone
> > > >
> > > > On 2013-02-03, at 8:00 PM, "anil gupta" <an...@gmail.com> wrote:
> > > >
> > > > > Hi Dalia,
> > > > >
> > > > > Recently, i ran into same problem with YCSB. You will need to
> > recompile
> > > > > YCSB with HBase0.92. Download the maven project from yscb site and
> > > > > recompile it with HBase0.92.
> > > > >
> > > > > HTH,
> > > > > Anil Gupta
> > > > >
> > > > > On Sun, Feb 3, 2013 at 9:49 AM, Dalia Sobhy <
> > dalia.mohsobhy@hotmail.com
> > > > >wrote:
> > > > >
> > > > >>
> > > > >> Hello Mohamed,
> > > > >>
> > > > >> Yes Hbase is running good and already I have a table and and some
> > > > records
> > > > >> there.
> > > > >>
> > > > >> I have done what you told me I copied all jar files from lib
> > directory
> > > > to
> > > > >> YSCB/db/hbase/lib/ directory. After restart and writing this command
> > > > >> "./bin/ycsb run hbase -p columnfamily=family -P
> > workloads/workloada", I
> > > > >> receive the same error.
> > > > >>
> > > > >> But I didn't compile, because already I don't have a .java file
> > > > >>
> > > > >> ycsb-0.1.4 directory doesn't contain hbase client java file, it
> > contains
> > > > >> only this "hbase-binding-0.1.4.jar"
> > > > >>
> > > > >> So what to do ?
> > > > >>
> > > > >>
> > > > >>> From: dontariq@gmail.com
> > > > >>> Date: Sun, 3 Feb 2013 22:54:13 +0530
> > > > >>> Subject: Re: Hbase Read/Write throughput measure
> > > > >>> To: user@hbase.apache.org
> > > > >>>
> > > > >>> Hello Dalia,
> > > > >>>
> > > > >>>        Is your Hbase running OK? Copy the hbase jar from your
> > > > HBASE_HOME
> > > > >>> and all the jars from your HBASE_HOME/lib/ directory to your
> > > > >>> YSCB/db/hbase/lib/ directory, compile it again and restart
> > everything.
> > > > >>>
> > > > >>> Warm Regards,
> > > > >>> Tariq
> > > > >>> https://mtariq.jux.com/
> > > > >>> cloudfront.blogspot.com
> > > > >>>
> > > > >>>
> > > > >>> On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <
> > > > dalia.mohsobhy@hotmail.com
> > > > >>> wrote:
> > > > >>>
> > > > >>>>
> > > > >>>> Hello Mohamed,
> > > > >>>>
> > > > >>>> I believe thats not the case.
> > > > >>>>
> > > > >>>> check this blog
> > > > >>
> > > >
> > http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
> > > > >>>>
> > > > >>>> It denotes that this error is due to incompatible Hbase client and
> > > > >> server
> > > > >>>> versions, although I am using Hbase server :0.92 and the server
> > is the
> > > > >>>> latest one uploaded by ycsb. So I don't know why this is
> > happening.
> > > > >>>>
> > > > >>>> So do you have any idea?
> > > > >>>>
> > > > >>>> This is the cmd I am executing:
> > > > >>>>
> > > > >>>> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p
> > > > >> columnfamily="family"
> > > > >>>> -P workloads/workloada
> > > > >>>>
> > > > >>>> java -cp
> > > > >>
> > > >
> > /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > > > >>>> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
> > > > >>>> columnfamily=family -P workloads/workloada -t
> > > > >>>> YCSB Client 0.1
> > > > >>>> Command line: -db com.yahoo.ycsb.db.HBaseClient -p
> > columnfamily=family
> > > > >> -P
> > > > >>>> workloads/workloada -t
> > > > >>>> Loading workload...
> > > > >>>> Starting test.
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>>> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010
> > 05:32
> > > > >> GMT
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > > > >> host.name=
> > > > >>>> master1.medcloud.com
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>>> environment:java.version=1.6.0_31
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>>> environment:java.vendor=Sun Microsystems Inc.
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>>> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>
> > > >
> > environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>
> > > >
> > environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>>> environment:java.io.tmpdir=/tmp
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>>> environment:java.compiler=<NA>
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > > > os.name
> > > > >>>> =Linux
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>>> environment:os.arch=amd64
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>>> environment:os.version=3.2.0-23-generic
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > > > >> user.name
> > > > >>>> =root
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>>> environment:user.home=/root
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > > >>>> environment:user.dir=/home/dalia/ycsb-0.1.4
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client
> > > > >> connection,
> > > > >>>> connectString=192.168.2.3:2181 sessionTimeout=180000
> > > > >> watcher=hconnection
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket
> > connection
> > > > >> to
> > > > >>>> server /192.168.2.3:2181
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection
> > > > >> established
> > > > >>>> to master1.medcloud.com/192.168.2.3:2181, initiating session
> > > > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
> > > > >>>> complete on server master1.medcloud.com/192.168.2.3:2181,
> > sessionid =
> > > > >>>> 0x13c9b45ac70000a, negotiated timeout = 60000
> > > > >>>> java.lang.IllegalArgumentException: Not a host:port pair: �
> > > > >>>> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
> > > > >>>>
> > > > >>>>
> > > > >>>> my hbase-site.xml:
> > > > >>>>
> > > > >>>> <configuration>
> > > > >>>>  <property>
> > > > >>>>    <name>hbase.rootdir</name>
> > > > >>>>   <value>hdfs://192.168.2.3:20001/hbase</value>
> > > > >>>>
> > > > >>>> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
> > > > >>>>    <description>The directory shared by region servers.
> > > > >>>>    </description>
> > > > >>>>  </property>
> > > > >>>>  <property>
> > > > >>>>    <name>hbase.master</name>
> > > > >>>>        <value>192.168.2.3:60000</value>
> > > > >>>>    <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
> > > > >>>>    <description>The host and port that the HBase master runs at.
> > > > >>>>    </description>
> > > > >>>>  </property>
> > > > >>>>  <property>
> > > > >>>>    <name>hbase.zookeeper.quorum</name>
> > > > >>>>    <value>192.168.2.3</value>
> > > > >>>>    <!--<value><HBASE_MASTER_SERVER></value>-->
> > > > >>>>  </property>
> > > > >>>> </configuration>
> > > > >>>>
> > > > >>>> So any help please....
> > > > >>>>
> > > > >>>> Thanks,
> > > > >>>>
> > > > >>>>> From: dontariq@gmail.com
> > > > >>>>> Date: Sun, 3 Feb 2013 13:54:28 +0530
> > > > >>>>> Subject: Re: Hbase Read/Write throughput measure
> > > > >>>>> To: user@hbase.apache.org
> > > > >>>>>
> > > > >>>>> Hello Dalia,
> > > > >>>>>
> > > > >>>>>       I am sorry for late response, courtesy hectic schedule. Add
> > > > >> the
> > > > >>>>> hbase-site.xml file to the classpath and see if it works for you.
> > > > >>>>>
> > > > >>>>> Warm Regards,
> > > > >>>>> Tariq
> > > > >>>>> https://mtariq.jux.com/
> > > > >>>>> cloudfront.blogspot.com
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <
> > > > >> dalia.mohsobhy@hotmail.com
> > > > >>>>> wrote:
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>> I tried yahoo bench mark I got the following error:
> > > > >>>>>>
> > > > >>>>>> java.lang.IllegalArgumentException: Not a host:port
> > > > >>>>>>
> > > > >>>>>> So any help,
> > > > >>>>>>
> > > > >>>>>> Note that I am using hbase 0.92
> > > > >>>>>>
> > > > >>>>>> I have downloaded the lastest ycsb from github
> > > > >>>>>>
> > > > >>>>>>> From: dalia.mohsobhy@hotmail.com
> > > > >>>>>>> To: user@hbase.apache.org
> > > > >>>>>>> Subject: RE: Hbase Read/Write throughput measure
> > > > >>>>>>> Date: Sat, 2 Feb 2013 16:47:43 +0200
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> any help ??!!!!!!
> > > > >>>>>>>
> > > > >>>>>>>> Date: Fri, 1 Feb 2013 17:15:45 -0500
> > > > >>>>>>>> Subject: Re: Hbase Read/Write throughput measure
> > > > >>>>>>>> From: jean-marc@spaggiari.org
> > > > >>>>>>>> To: user@hbase.apache.org
> > > > >>>>>>>>
> > > > >>>>>>>> When you will have figure how the command is working, you will
> > > > >>>> have to
> > > > >>>>>>>> understand the output ;)
> > > > >>>>>>>>
> > > > >>>>>>>> This is how I just tried it from my HBase directory:
> > > > >>>>>>>> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
> > > > >> randomRead
> > > > >>>> 10
> > > > >>>>>>>>
> > > > >>>>>>>> And this is the output (last lines):
> > > > >>>>>>>> 10066326    56314
> > > > >>>>>>>> 10171183    56385
> > > > >>>>>>>> 10276040    56579
> > > > >>>>>>>> 10380897    56553
> > > > >>>>>>>>
> > > > >>>>>>>> Does ask me what it means ;) I have, so far, no idea.
> > > > >>>>>>>>
> > > > >>>>>>>> JM
> > > > >>>>>>>>
> > > > >>>>>>>> 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> > > > >>>>>>>>> Dear Mohamed,
> > > > >>>>>>>>>
> > > > >>>>>>>>> I checked this link:
> > > > >>>>>>>>> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > > > >>>>>>>>>
> > > > >>>>>>>>> But how to use the commands there is no clue, note that I am
> > > > >>>> using
> > > > >>>>>> cloudera
> > > > >>>>>>>>> manager.
> > > > >>>>>>>>>
> > > > >>>>>>>>> The other yahoo link I couldn't understand how to use it.
> > > > >>>>>>>>>
> > > > >>>>>>>>> So any help please??
> > > > >>>>>>>>>
> > > > >>>>>>>>> Sent from my iPad
> > > > >>>>>>>>>
> > > > >>>>>>>>> On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <
> > > > >> dontariq@gmail.com
> > > > >>>>>
> > > > >>>>>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>>> Hello Dalia,
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>         I think the easiest way to measure the read/write
> > > > >>>>>> throughput is
> > > > >>>>>>>>>> to use "PerformanceEvaluation" tool that comes with the
> > > > >> Hbase
> > > > >>>>>>>>>> distribution. It spawns a map-reduce job to do the
> > > > >> reads/writes
> > > > >>>> in
> > > > >>>>>>>>>> parallel. Apart from this there are several other ways to
> > > > >>>> benchmark
> > > > >>>>>> your
> > > > >>>>>>>>>> Hbase cluster like YSCB <
> > > > >>>> https://github.com/brianfrankcooper/YCSB>.
> > > > >>>>>> You
> > > > >>>>>>>>>> might find this
> > > > >>>>>>>>>> link<
> > > > >> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> > > > >>>>>> useful
> > > > >>>>>>>>>> that talks about Hbase performance.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> HTH
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Warm Regards,
> > > > >>>>>>>>>> Tariq
> > > > >>>>>>>>>> https://mtariq.jux.com/
> > > > >>>>>>>>>> cloudfront.blogspot.com
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > > > >>>>>>>>>> <da...@hotmail.com>wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Dear all,
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> I want to measure the read/write throughput for a code on a
> > > > >>>>>> cluster of
> > > > >>>>>>>>>>> 10
> > > > >>>>>>>>>>> nodes. So is there any code or way to measure it?
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> I have seen in a cloudera-based presentation that hbase
> > > > >>>> read/write
> > > > >>>>>>>>>>> throughput = millions queries per second.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> So any help please??
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Thanks
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Best Regards,
> > > > >>>>>>>>>>> Dalia Sobhy
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks & Regards,
> > > > > Anil Gupta
> > > >
> >
> >

Re: Hbase Read/Write throughput measure

Posted by Mohammad Tariq <do...@gmail.com>.

it may take some time depending on your connection. wait for sometime. if
you still find some problem you can visit this
link<http://cloudfront.blogspot.in/2013/02/how-to-benchmark-hbase-using-ycsb.html#.UQ7Tu5vS8s0>

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Mon, Feb 4, 2013 at 2:35 AM, Dalia Sobhy <da...@hotmail.com>wrote:

>
> Hello Mohamed,
>
> I am using ubuntu 12, but the problem is that I was facing some errors
> while compiling, after solving them.
>
> After $git clone http://github.com/brianfrankcooper/YCSB.git
> Cloning into 'YCSB'...
> remote: Counting objects: 2260, done.
> remote: Compressing objects: 100% (694/694), done.
> Receiving objects:  86% (1952/2260), 764.00 KiB | 51 KiB/s
>
> It keeps hanging on 86%, so any clue??
>
> Thanksss :D
>
> > From: dontariq@gmail.com
> > Date: Mon, 4 Feb 2013 00:17:40 +0530
> > Subject: Re: Hbase Read/Write throughput measure
> > To: user@hbase.apache.org
> >
> > Hello Dalia,
> >
> >           Me and Anil are talking about this :
> >
> > 1 - git clone http://github.com/brianfrankcooper/YCSB.git
> > 2 - cd YCSB/
> > 3 - mvn compile
> >
> > And not compiling through Java
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
> >
> > On Mon, Feb 4, 2013 at 12:14 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
> >wrote:
> >
> > > Hi anil,
> > >
> > > How to recompile it ?
> > >
> > > Sent from my iPhone
> > >
> > > On 2013-02-03, at 8:00 PM, "anil gupta" <an...@gmail.com> wrote:
> > >
> > > > Hi Dalia,
> > > >
> > > > Recently, i ran into same problem with YCSB. You will need to
> recompile
> > > > YCSB with HBase0.92. Download the maven project from yscb site and
> > > > recompile it with HBase0.92.
> > > >
> > > > HTH,
> > > > Anil Gupta
> > > >
> > > > On Sun, Feb 3, 2013 at 9:49 AM, Dalia Sobhy <
> dalia.mohsobhy@hotmail.com
> > > >wrote:
> > > >
> > > >>
> > > >> Hello Mohamed,
> > > >>
> > > >> Yes Hbase is running good and already I have a table and and some
> > > records
> > > >> there.
> > > >>
> > > >> I have done what you told me I copied all jar files from lib
> directory
> > > to
> > > >> YSCB/db/hbase/lib/ directory. After restart and writing this command
> > > >> "./bin/ycsb run hbase -p columnfamily=family -P
> workloads/workloada", I
> > > >> receive the same error.
> > > >>
> > > >> But I didn't compile, because already I don't have a .java file
> > > >>
> > > >> ycsb-0.1.4 directory doesn't contain hbase client java file, it
> contains
> > > >> only this "hbase-binding-0.1.4.jar"
> > > >>
> > > >> So what to do ?
> > > >>
> > > >>
> > > >>> From: dontariq@gmail.com
> > > >>> Date: Sun, 3 Feb 2013 22:54:13 +0530
> > > >>> Subject: Re: Hbase Read/Write throughput measure
> > > >>> To: user@hbase.apache.org
> > > >>>
> > > >>> Hello Dalia,
> > > >>>
> > > >>>        Is your Hbase running OK? Copy the hbase jar from your
> > > HBASE_HOME
> > > >>> and all the jars from your HBASE_HOME/lib/ directory to your
> > > >>> YSCB/db/hbase/lib/ directory, compile it again and restart
> everything.
> > > >>>
> > > >>> Warm Regards,
> > > >>> Tariq
> > > >>> https://mtariq.jux.com/
> > > >>> cloudfront.blogspot.com
> > > >>>
> > > >>>
> > > >>> On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <
> > > dalia.mohsobhy@hotmail.com
> > > >>> wrote:
> > > >>>
> > > >>>>
> > > >>>> Hello Mohamed,
> > > >>>>
> > > >>>> I believe thats not the case.
> > > >>>>
> > > >>>> check this blog
> > > >>
> > >
> http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
> > > >>>>
> > > >>>> It denotes that this error is due to incompatible Hbase client and
> > > >> server
> > > >>>> versions, although I am using Hbase server :0.92 and the server
> is the
> > > >>>> latest one uploaded by ycsb. So I don't know why this is
> happening.
> > > >>>>
> > > >>>> So do you have any idea?
> > > >>>>
> > > >>>> This is the cmd I am executing:
> > > >>>>
> > > >>>> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p
> > > >> columnfamily="family"
> > > >>>> -P workloads/workloada
> > > >>>>
> > > >>>> java -cp
> > > >>
> > >
> /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > > >>>> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
> > > >>>> columnfamily=family -P workloads/workloada -t
> > > >>>> YCSB Client 0.1
> > > >>>> Command line: -db com.yahoo.ycsb.db.HBaseClient -p
> columnfamily=family
> > > >> -P
> > > >>>> workloads/workloada -t
> > > >>>> Loading workload...
> > > >>>> Starting test.
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010
> 05:32
> > > >> GMT
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > > >> host.name=
> > > >>>> master1.medcloud.com
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:java.version=1.6.0_31
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:java.vendor=Sun Microsystems Inc.
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>
> > >
> environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>
> > >
> environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:java.io.tmpdir=/tmp
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:java.compiler=<NA>
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > > os.name
> > > >>>> =Linux
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:os.arch=amd64
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:os.version=3.2.0-23-generic
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > > >> user.name
> > > >>>> =root
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:user.home=/root
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:user.dir=/home/dalia/ycsb-0.1.4
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client
> > > >> connection,
> > > >>>> connectString=192.168.2.3:2181 sessionTimeout=180000
> > > >> watcher=hconnection
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket
> connection
> > > >> to
> > > >>>> server /192.168.2.3:2181
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection
> > > >> established
> > > >>>> to master1.medcloud.com/192.168.2.3:2181, initiating session
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
> > > >>>> complete on server master1.medcloud.com/192.168.2.3:2181,
> sessionid =
> > > >>>> 0x13c9b45ac70000a, negotiated timeout = 60000
> > > >>>> java.lang.IllegalArgumentException: Not a host:port pair: �
> > > >>>> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
> > > >>>>
> > > >>>>
> > > >>>> my hbase-site.xml:
> > > >>>>
> > > >>>> <configuration>
> > > >>>>  <property>
> > > >>>>    <name>hbase.rootdir</name>
> > > >>>>   <value>hdfs://192.168.2.3:20001/hbase</value>
> > > >>>>
> > > >>>> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
> > > >>>>    <description>The directory shared by region servers.
> > > >>>>    </description>
> > > >>>>  </property>
> > > >>>>  <property>
> > > >>>>    <name>hbase.master</name>
> > > >>>>        <value>192.168.2.3:60000</value>
> > > >>>>    <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
> > > >>>>    <description>The host and port that the HBase master runs at.
> > > >>>>    </description>
> > > >>>>  </property>
> > > >>>>  <property>
> > > >>>>    <name>hbase.zookeeper.quorum</name>
> > > >>>>    <value>192.168.2.3</value>
> > > >>>>    <!--<value><HBASE_MASTER_SERVER></value>-->
> > > >>>>  </property>
> > > >>>> </configuration>
> > > >>>>
> > > >>>> So any help please....
> > > >>>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>>> From: dontariq@gmail.com
> > > >>>>> Date: Sun, 3 Feb 2013 13:54:28 +0530
> > > >>>>> Subject: Re: Hbase Read/Write throughput measure
> > > >>>>> To: user@hbase.apache.org
> > > >>>>>
> > > >>>>> Hello Dalia,
> > > >>>>>
> > > >>>>>       I am sorry for late response, courtesy hectic schedule. Add
> > > >> the
> > > >>>>> hbase-site.xml file to the classpath and see if it works for you.
> > > >>>>>
> > > >>>>> Warm Regards,
> > > >>>>> Tariq
> > > >>>>> https://mtariq.jux.com/
> > > >>>>> cloudfront.blogspot.com
> > > >>>>>
> > > >>>>>
> > > >>>>> On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <
> > > >> dalia.mohsobhy@hotmail.com
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>>>
> > > >>>>>> I tried yahoo bench mark I got the following error:
> > > >>>>>>
> > > >>>>>> java.lang.IllegalArgumentException: Not a host:port
> > > >>>>>>
> > > >>>>>> So any help,
> > > >>>>>>
> > > >>>>>> Note that I am using hbase 0.92
> > > >>>>>>
> > > >>>>>> I have downloaded the lastest ycsb from github
> > > >>>>>>
> > > >>>>>>> From: dalia.mohsobhy@hotmail.com
> > > >>>>>>> To: user@hbase.apache.org
> > > >>>>>>> Subject: RE: Hbase Read/Write throughput measure
> > > >>>>>>> Date: Sat, 2 Feb 2013 16:47:43 +0200
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> any help ??!!!!!!
> > > >>>>>>>
> > > >>>>>>>> Date: Fri, 1 Feb 2013 17:15:45 -0500
> > > >>>>>>>> Subject: Re: Hbase Read/Write throughput measure
> > > >>>>>>>> From: jean-marc@spaggiari.org
> > > >>>>>>>> To: user@hbase.apache.org
> > > >>>>>>>>
> > > >>>>>>>> When you will have figure how the command is working, you will
> > > >>>> have to
> > > >>>>>>>> understand the output ;)
> > > >>>>>>>>
> > > >>>>>>>> This is how I just tried it from my HBase directory:
> > > >>>>>>>> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
> > > >> randomRead
> > > >>>> 10
> > > >>>>>>>>
> > > >>>>>>>> And this is the output (last lines):
> > > >>>>>>>> 10066326    56314
> > > >>>>>>>> 10171183    56385
> > > >>>>>>>> 10276040    56579
> > > >>>>>>>> 10380897    56553
> > > >>>>>>>>
> > > >>>>>>>> Does ask me what it means ;) I have, so far, no idea.
> > > >>>>>>>>
> > > >>>>>>>> JM
> > > >>>>>>>>
> > > >>>>>>>> 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> > > >>>>>>>>> Dear Mohamed,
> > > >>>>>>>>>
> > > >>>>>>>>> I checked this link:
> > > >>>>>>>>> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > > >>>>>>>>>
> > > >>>>>>>>> But how to use the commands there is no clue, note that I am
> > > >>>> using
> > > >>>>>> cloudera
> > > >>>>>>>>> manager.
> > > >>>>>>>>>
> > > >>>>>>>>> The other yahoo link I couldn't understand how to use it.
> > > >>>>>>>>>
> > > >>>>>>>>> So any help please??
> > > >>>>>>>>>
> > > >>>>>>>>> Sent from my iPad
> > > >>>>>>>>>
> > > >>>>>>>>> On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <
> > > >> dontariq@gmail.com
> > > >>>>>
> > > >>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> Hello Dalia,
> > > >>>>>>>>>>
> > > >>>>>>>>>>         I think the easiest way to measure the read/write
> > > >>>>>> throughput is
> > > >>>>>>>>>> to use "PerformanceEvaluation" tool that comes with the
> > > >> Hbase
> > > >>>>>>>>>> distribution. It spawns a map-reduce job to do the
> > > >> reads/writes
> > > >>>> in
> > > >>>>>>>>>> parallel. Apart from this there are several other ways to
> > > >>>> benchmark
> > > >>>>>> your
> > > >>>>>>>>>> Hbase cluster like YSCB <
> > > >>>> https://github.com/brianfrankcooper/YCSB>.
> > > >>>>>> You
> > > >>>>>>>>>> might find this
> > > >>>>>>>>>> link<
> > > >> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> > > >>>>>> useful
> > > >>>>>>>>>> that talks about Hbase performance.
> > > >>>>>>>>>>
> > > >>>>>>>>>> HTH
> > > >>>>>>>>>>
> > > >>>>>>>>>> Warm Regards,
> > > >>>>>>>>>> Tariq
> > > >>>>>>>>>> https://mtariq.jux.com/
> > > >>>>>>>>>> cloudfront.blogspot.com
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > > >>>>>>>>>> <da...@hotmail.com>wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Dear all,
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> I want to measure the read/write throughput for a code on a
> > > >>>>>> cluster of
> > > >>>>>>>>>>> 10
> > > >>>>>>>>>>> nodes. So is there any code or way to measure it?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> I have seen in a cloudera-based presentation that hbase
> > > >>>> read/write
> > > >>>>>>>>>>> throughput = millions queries per second.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> So any help please??
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Thanks
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Best Regards,
> > > >>>>>>>>>>> Dalia Sobhy
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks & Regards,
> > > > Anil Gupta
> > >
>
>

RE: Hbase Read/Write throughput measure

Posted by Dalia Sobhy <da...@hotmail.com>.

Hello Mohamed,

I am using ubuntu 12, but the problem is that I was facing some errors while compiling, after solving them.

After $git clone http://github.com/brianfrankcooper/YCSB.git
Cloning into 'YCSB'...
remote: Counting objects: 2260, done.
remote: Compressing objects: 100% (694/694), done.
Receiving objects:  86% (1952/2260), 764.00 KiB | 51 KiB/s  

It keeps hanging on 86%, so any clue??

Thanksss :D

> From: dontariq@gmail.com
> Date: Mon, 4 Feb 2013 00:17:40 +0530
> Subject: Re: Hbase Read/Write throughput measure
> To: user@hbase.apache.org
> 
> Hello Dalia,
> 
>           Me and Anil are talking about this :
> 
> 1 - git clone http://github.com/brianfrankcooper/YCSB.git
> 2 - cd YCSB/
> 3 - mvn compile
> 
> And not compiling through Java
> 
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
> 
> 
> On Mon, Feb 4, 2013 at 12:14 AM, Dalia Sobhy <da...@hotmail.com>wrote:
> 
> > Hi anil,
> >
> > How to recompile it ?
> >
> > Sent from my iPhone
> >
> > On 2013-02-03, at 8:00 PM, "anil gupta" <an...@gmail.com> wrote:
> >
> > > Hi Dalia,
> > >
> > > Recently, i ran into same problem with YCSB. You will need to recompile
> > > YCSB with HBase0.92. Download the maven project from yscb site and
> > > recompile it with HBase0.92.
> > >
> > > HTH,
> > > Anil Gupta
> > >
> > > On Sun, Feb 3, 2013 at 9:49 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
> > >wrote:
> > >
> > >>
> > >> Hello Mohamed,
> > >>
> > >> Yes Hbase is running good and already I have a table and and some
> > records
> > >> there.
> > >>
> > >> I have done what you told me I copied all jar files from lib directory
> > to
> > >> YSCB/db/hbase/lib/ directory. After restart and writing this command
> > >> "./bin/ycsb run hbase -p columnfamily=family -P workloads/workloada", I
> > >> receive the same error.
> > >>
> > >> But I didn't compile, because already I don't have a .java file
> > >>
> > >> ycsb-0.1.4 directory doesn't contain hbase client java file, it contains
> > >> only this "hbase-binding-0.1.4.jar"
> > >>
> > >> So what to do ?
> > >>
> > >>
> > >>> From: dontariq@gmail.com
> > >>> Date: Sun, 3 Feb 2013 22:54:13 +0530
> > >>> Subject: Re: Hbase Read/Write throughput measure
> > >>> To: user@hbase.apache.org
> > >>>
> > >>> Hello Dalia,
> > >>>
> > >>>        Is your Hbase running OK? Copy the hbase jar from your
> > HBASE_HOME
> > >>> and all the jars from your HBASE_HOME/lib/ directory to your
> > >>> YSCB/db/hbase/lib/ directory, compile it again and restart everything.
> > >>>
> > >>> Warm Regards,
> > >>> Tariq
> > >>> https://mtariq.jux.com/
> > >>> cloudfront.blogspot.com
> > >>>
> > >>>
> > >>> On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <
> > dalia.mohsobhy@hotmail.com
> > >>> wrote:
> > >>>
> > >>>>
> > >>>> Hello Mohamed,
> > >>>>
> > >>>> I believe thats not the case.
> > >>>>
> > >>>> check this blog
> > >>
> > http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
> > >>>>
> > >>>> It denotes that this error is due to incompatible Hbase client and
> > >> server
> > >>>> versions, although I am using Hbase server :0.92 and the server is the
> > >>>> latest one uploaded by ycsb. So I don't know why this is happening.
> > >>>>
> > >>>> So do you have any idea?
> > >>>>
> > >>>> This is the cmd I am executing:
> > >>>>
> > >>>> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p
> > >> columnfamily="family"
> > >>>> -P workloads/workloada
> > >>>>
> > >>>> java -cp
> > >>
> > /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > >>>> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
> > >>>> columnfamily=family -P workloads/workloada -t
> > >>>> YCSB Client 0.1
> > >>>> Command line: -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family
> > >> -P
> > >>>> workloads/workloada -t
> > >>>> Loading workload...
> > >>>> Starting test.
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32
> > >> GMT
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > >> host.name=
> > >>>> master1.medcloud.com
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:java.version=1.6.0_31
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:java.vendor=Sun Microsystems Inc.
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>
> > environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>
> > environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:java.io.tmpdir=/tmp
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:java.compiler=<NA>
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > os.name
> > >>>> =Linux
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:os.arch=amd64
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:os.version=3.2.0-23-generic
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > >> user.name
> > >>>> =root
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:user.home=/root
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:user.dir=/home/dalia/ycsb-0.1.4
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client
> > >> connection,
> > >>>> connectString=192.168.2.3:2181 sessionTimeout=180000
> > >> watcher=hconnection
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket connection
> > >> to
> > >>>> server /192.168.2.3:2181
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection
> > >> established
> > >>>> to master1.medcloud.com/192.168.2.3:2181, initiating session
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
> > >>>> complete on server master1.medcloud.com/192.168.2.3:2181, sessionid =
> > >>>> 0x13c9b45ac70000a, negotiated timeout = 60000
> > >>>> java.lang.IllegalArgumentException: Not a host:port pair: �
> > >>>> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
> > >>>>
> > >>>>
> > >>>> my hbase-site.xml:
> > >>>>
> > >>>> <configuration>
> > >>>>  <property>
> > >>>>    <name>hbase.rootdir</name>
> > >>>>   <value>hdfs://192.168.2.3:20001/hbase</value>
> > >>>>
> > >>>> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
> > >>>>    <description>The directory shared by region servers.
> > >>>>    </description>
> > >>>>  </property>
> > >>>>  <property>
> > >>>>    <name>hbase.master</name>
> > >>>>        <value>192.168.2.3:60000</value>
> > >>>>    <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
> > >>>>    <description>The host and port that the HBase master runs at.
> > >>>>    </description>
> > >>>>  </property>
> > >>>>  <property>
> > >>>>    <name>hbase.zookeeper.quorum</name>
> > >>>>    <value>192.168.2.3</value>
> > >>>>    <!--<value><HBASE_MASTER_SERVER></value>-->
> > >>>>  </property>
> > >>>> </configuration>
> > >>>>
> > >>>> So any help please....
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>>> From: dontariq@gmail.com
> > >>>>> Date: Sun, 3 Feb 2013 13:54:28 +0530
> > >>>>> Subject: Re: Hbase Read/Write throughput measure
> > >>>>> To: user@hbase.apache.org
> > >>>>>
> > >>>>> Hello Dalia,
> > >>>>>
> > >>>>>       I am sorry for late response, courtesy hectic schedule. Add
> > >> the
> > >>>>> hbase-site.xml file to the classpath and see if it works for you.
> > >>>>>
> > >>>>> Warm Regards,
> > >>>>> Tariq
> > >>>>> https://mtariq.jux.com/
> > >>>>> cloudfront.blogspot.com
> > >>>>>
> > >>>>>
> > >>>>> On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <
> > >> dalia.mohsobhy@hotmail.com
> > >>>>> wrote:
> > >>>>>
> > >>>>>>
> > >>>>>> I tried yahoo bench mark I got the following error:
> > >>>>>>
> > >>>>>> java.lang.IllegalArgumentException: Not a host:port
> > >>>>>>
> > >>>>>> So any help,
> > >>>>>>
> > >>>>>> Note that I am using hbase 0.92
> > >>>>>>
> > >>>>>> I have downloaded the lastest ycsb from github
> > >>>>>>
> > >>>>>>> From: dalia.mohsobhy@hotmail.com
> > >>>>>>> To: user@hbase.apache.org
> > >>>>>>> Subject: RE: Hbase Read/Write throughput measure
> > >>>>>>> Date: Sat, 2 Feb 2013 16:47:43 +0200
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> any help ??!!!!!!
> > >>>>>>>
> > >>>>>>>> Date: Fri, 1 Feb 2013 17:15:45 -0500
> > >>>>>>>> Subject: Re: Hbase Read/Write throughput measure
> > >>>>>>>> From: jean-marc@spaggiari.org
> > >>>>>>>> To: user@hbase.apache.org
> > >>>>>>>>
> > >>>>>>>> When you will have figure how the command is working, you will
> > >>>> have to
> > >>>>>>>> understand the output ;)
> > >>>>>>>>
> > >>>>>>>> This is how I just tried it from my HBase directory:
> > >>>>>>>> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
> > >> randomRead
> > >>>> 10
> > >>>>>>>>
> > >>>>>>>> And this is the output (last lines):
> > >>>>>>>> 10066326    56314
> > >>>>>>>> 10171183    56385
> > >>>>>>>> 10276040    56579
> > >>>>>>>> 10380897    56553
> > >>>>>>>>
> > >>>>>>>> Does ask me what it means ;) I have, so far, no idea.
> > >>>>>>>>
> > >>>>>>>> JM
> > >>>>>>>>
> > >>>>>>>> 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> > >>>>>>>>> Dear Mohamed,
> > >>>>>>>>>
> > >>>>>>>>> I checked this link:
> > >>>>>>>>> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > >>>>>>>>>
> > >>>>>>>>> But how to use the commands there is no clue, note that I am
> > >>>> using
> > >>>>>> cloudera
> > >>>>>>>>> manager.
> > >>>>>>>>>
> > >>>>>>>>> The other yahoo link I couldn't understand how to use it.
> > >>>>>>>>>
> > >>>>>>>>> So any help please??
> > >>>>>>>>>
> > >>>>>>>>> Sent from my iPad
> > >>>>>>>>>
> > >>>>>>>>> On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <
> > >> dontariq@gmail.com
> > >>>>>
> > >>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hello Dalia,
> > >>>>>>>>>>
> > >>>>>>>>>>         I think the easiest way to measure the read/write
> > >>>>>> throughput is
> > >>>>>>>>>> to use "PerformanceEvaluation" tool that comes with the
> > >> Hbase
> > >>>>>>>>>> distribution. It spawns a map-reduce job to do the
> > >> reads/writes
> > >>>> in
> > >>>>>>>>>> parallel. Apart from this there are several other ways to
> > >>>> benchmark
> > >>>>>> your
> > >>>>>>>>>> Hbase cluster like YSCB <
> > >>>> https://github.com/brianfrankcooper/YCSB>.
> > >>>>>> You
> > >>>>>>>>>> might find this
> > >>>>>>>>>> link<
> > >> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> > >>>>>> useful
> > >>>>>>>>>> that talks about Hbase performance.
> > >>>>>>>>>>
> > >>>>>>>>>> HTH
> > >>>>>>>>>>
> > >>>>>>>>>> Warm Regards,
> > >>>>>>>>>> Tariq
> > >>>>>>>>>> https://mtariq.jux.com/
> > >>>>>>>>>> cloudfront.blogspot.com
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > >>>>>>>>>> <da...@hotmail.com>wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Dear all,
> > >>>>>>>>>>>
> > >>>>>>>>>>> I want to measure the read/write throughput for a code on a
> > >>>>>> cluster of
> > >>>>>>>>>>> 10
> > >>>>>>>>>>> nodes. So is there any code or way to measure it?
> > >>>>>>>>>>>
> > >>>>>>>>>>> I have seen in a cloudera-based presentation that hbase
> > >>>> read/write
> > >>>>>>>>>>> throughput = millions queries per second.
> > >>>>>>>>>>>
> > >>>>>>>>>>> So any help please??
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Best Regards,
> > >>>>>>>>>>> Dalia Sobhy
> > >
> > >
> > >
> > > --
> > > Thanks & Regards,
> > > Anil Gupta
> >

Re: Hbase Read/Write throughput measure

Posted by Mohammad Tariq <do...@gmail.com>.

Hello Dalia,

          Me and Anil are talking about this :

1 - git clone http://github.com/brianfrankcooper/YCSB.git
2 - cd YCSB/
3 - mvn compile

And not compiling through Java

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Mon, Feb 4, 2013 at 12:14 AM, Dalia Sobhy <da...@hotmail.com>wrote:

> Hi anil,
>
> How to recompile it ?
>
> Sent from my iPhone
>
> On 2013-02-03, at 8:00 PM, "anil gupta" <an...@gmail.com> wrote:
>
> > Hi Dalia,
> >
> > Recently, i ran into same problem with YCSB. You will need to recompile
> > YCSB with HBase0.92. Download the maven project from yscb site and
> > recompile it with HBase0.92.
> >
> > HTH,
> > Anil Gupta
> >
> > On Sun, Feb 3, 2013 at 9:49 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
> >wrote:
> >
> >>
> >> Hello Mohamed,
> >>
> >> Yes Hbase is running good and already I have a table and and some
> records
> >> there.
> >>
> >> I have done what you told me I copied all jar files from lib directory
> to
> >> YSCB/db/hbase/lib/ directory. After restart and writing this command
> >> "./bin/ycsb run hbase -p columnfamily=family -P workloads/workloada", I
> >> receive the same error.
> >>
> >> But I didn't compile, because already I don't have a .java file
> >>
> >> ycsb-0.1.4 directory doesn't contain hbase client java file, it contains
> >> only this "hbase-binding-0.1.4.jar"
> >>
> >> So what to do ?
> >>
> >>
> >>> From: dontariq@gmail.com
> >>> Date: Sun, 3 Feb 2013 22:54:13 +0530
> >>> Subject: Re: Hbase Read/Write throughput measure
> >>> To: user@hbase.apache.org
> >>>
> >>> Hello Dalia,
> >>>
> >>>        Is your Hbase running OK? Copy the hbase jar from your
> HBASE_HOME
> >>> and all the jars from your HBASE_HOME/lib/ directory to your
> >>> YSCB/db/hbase/lib/ directory, compile it again and restart everything.
> >>>
> >>> Warm Regards,
> >>> Tariq
> >>> https://mtariq.jux.com/
> >>> cloudfront.blogspot.com
> >>>
> >>>
> >>> On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <
> dalia.mohsobhy@hotmail.com
> >>> wrote:
> >>>
> >>>>
> >>>> Hello Mohamed,
> >>>>
> >>>> I believe thats not the case.
> >>>>
> >>>> check this blog
> >>
> http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
> >>>>
> >>>> It denotes that this error is due to incompatible Hbase client and
> >> server
> >>>> versions, although I am using Hbase server :0.92 and the server is the
> >>>> latest one uploaded by ycsb. So I don't know why this is happening.
> >>>>
> >>>> So do you have any idea?
> >>>>
> >>>> This is the cmd I am executing:
> >>>>
> >>>> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p
> >> columnfamily="family"
> >>>> -P workloads/workloada
> >>>>
> >>>> java -cp
> >>
> /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> >>>> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
> >>>> columnfamily=family -P workloads/workloada -t
> >>>> YCSB Client 0.1
> >>>> Command line: -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family
> >> -P
> >>>> workloads/workloada -t
> >>>> Loading workload...
> >>>> Starting test.
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>>> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32
> >> GMT
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> >> host.name=
> >>>> master1.medcloud.com
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>>> environment:java.version=1.6.0_31
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>>> environment:java.vendor=Sun Microsystems Inc.
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>>> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>
> environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>
> environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>>> environment:java.io.tmpdir=/tmp
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>>> environment:java.compiler=<NA>
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> os.name
> >>>> =Linux
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>>> environment:os.arch=amd64
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>>> environment:os.version=3.2.0-23-generic
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> >> user.name
> >>>> =root
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>>> environment:user.home=/root
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> >>>> environment:user.dir=/home/dalia/ycsb-0.1.4
> >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client
> >> connection,
> >>>> connectString=192.168.2.3:2181 sessionTimeout=180000
> >> watcher=hconnection
> >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket connection
> >> to
> >>>> server /192.168.2.3:2181
> >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection
> >> established
> >>>> to master1.medcloud.com/192.168.2.3:2181, initiating session
> >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
> >>>> complete on server master1.medcloud.com/192.168.2.3:2181, sessionid =
> >>>> 0x13c9b45ac70000a, negotiated timeout = 60000
> >>>> java.lang.IllegalArgumentException: Not a host:port pair: �
> >>>> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
> >>>>
> >>>>
> >>>> my hbase-site.xml:
> >>>>
> >>>> <configuration>
> >>>>  <property>
> >>>>    <name>hbase.rootdir</name>
> >>>>   <value>hdfs://192.168.2.3:20001/hbase</value>
> >>>>
> >>>> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
> >>>>    <description>The directory shared by region servers.
> >>>>    </description>
> >>>>  </property>
> >>>>  <property>
> >>>>    <name>hbase.master</name>
> >>>>        <value>192.168.2.3:60000</value>
> >>>>    <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
> >>>>    <description>The host and port that the HBase master runs at.
> >>>>    </description>
> >>>>  </property>
> >>>>  <property>
> >>>>    <name>hbase.zookeeper.quorum</name>
> >>>>    <value>192.168.2.3</value>
> >>>>    <!--<value><HBASE_MASTER_SERVER></value>-->
> >>>>  </property>
> >>>> </configuration>
> >>>>
> >>>> So any help please....
> >>>>
> >>>> Thanks,
> >>>>
> >>>>> From: dontariq@gmail.com
> >>>>> Date: Sun, 3 Feb 2013 13:54:28 +0530
> >>>>> Subject: Re: Hbase Read/Write throughput measure
> >>>>> To: user@hbase.apache.org
> >>>>>
> >>>>> Hello Dalia,
> >>>>>
> >>>>>       I am sorry for late response, courtesy hectic schedule. Add
> >> the
> >>>>> hbase-site.xml file to the classpath and see if it works for you.
> >>>>>
> >>>>> Warm Regards,
> >>>>> Tariq
> >>>>> https://mtariq.jux.com/
> >>>>> cloudfront.blogspot.com
> >>>>>
> >>>>>
> >>>>> On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <
> >> dalia.mohsobhy@hotmail.com
> >>>>> wrote:
> >>>>>
> >>>>>>
> >>>>>> I tried yahoo bench mark I got the following error:
> >>>>>>
> >>>>>> java.lang.IllegalArgumentException: Not a host:port
> >>>>>>
> >>>>>> So any help,
> >>>>>>
> >>>>>> Note that I am using hbase 0.92
> >>>>>>
> >>>>>> I have downloaded the lastest ycsb from github
> >>>>>>
> >>>>>>> From: dalia.mohsobhy@hotmail.com
> >>>>>>> To: user@hbase.apache.org
> >>>>>>> Subject: RE: Hbase Read/Write throughput measure
> >>>>>>> Date: Sat, 2 Feb 2013 16:47:43 +0200
> >>>>>>>
> >>>>>>>
> >>>>>>> any help ??!!!!!!
> >>>>>>>
> >>>>>>>> Date: Fri, 1 Feb 2013 17:15:45 -0500
> >>>>>>>> Subject: Re: Hbase Read/Write throughput measure
> >>>>>>>> From: jean-marc@spaggiari.org
> >>>>>>>> To: user@hbase.apache.org
> >>>>>>>>
> >>>>>>>> When you will have figure how the command is working, you will
> >>>> have to
> >>>>>>>> understand the output ;)
> >>>>>>>>
> >>>>>>>> This is how I just tried it from my HBase directory:
> >>>>>>>> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
> >> randomRead
> >>>> 10
> >>>>>>>>
> >>>>>>>> And this is the output (last lines):
> >>>>>>>> 10066326    56314
> >>>>>>>> 10171183    56385
> >>>>>>>> 10276040    56579
> >>>>>>>> 10380897    56553
> >>>>>>>>
> >>>>>>>> Does ask me what it means ;) I have, so far, no idea.
> >>>>>>>>
> >>>>>>>> JM
> >>>>>>>>
> >>>>>>>> 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> >>>>>>>>> Dear Mohamed,
> >>>>>>>>>
> >>>>>>>>> I checked this link:
> >>>>>>>>> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> >>>>>>>>>
> >>>>>>>>> But how to use the commands there is no clue, note that I am
> >>>> using
> >>>>>> cloudera
> >>>>>>>>> manager.
> >>>>>>>>>
> >>>>>>>>> The other yahoo link I couldn't understand how to use it.
> >>>>>>>>>
> >>>>>>>>> So any help please??
> >>>>>>>>>
> >>>>>>>>> Sent from my iPad
> >>>>>>>>>
> >>>>>>>>> On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <
> >> dontariq@gmail.com
> >>>>>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hello Dalia,
> >>>>>>>>>>
> >>>>>>>>>>         I think the easiest way to measure the read/write
> >>>>>> throughput is
> >>>>>>>>>> to use "PerformanceEvaluation" tool that comes with the
> >> Hbase
> >>>>>>>>>> distribution. It spawns a map-reduce job to do the
> >> reads/writes
> >>>> in
> >>>>>>>>>> parallel. Apart from this there are several other ways to
> >>>> benchmark
> >>>>>> your
> >>>>>>>>>> Hbase cluster like YSCB <
> >>>> https://github.com/brianfrankcooper/YCSB>.
> >>>>>> You
> >>>>>>>>>> might find this
> >>>>>>>>>> link<
> >> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> >>>>>> useful
> >>>>>>>>>> that talks about Hbase performance.
> >>>>>>>>>>
> >>>>>>>>>> HTH
> >>>>>>>>>>
> >>>>>>>>>> Warm Regards,
> >>>>>>>>>> Tariq
> >>>>>>>>>> https://mtariq.jux.com/
> >>>>>>>>>> cloudfront.blogspot.com
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> >>>>>>>>>> <da...@hotmail.com>wrote:
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Dear all,
> >>>>>>>>>>>
> >>>>>>>>>>> I want to measure the read/write throughput for a code on a
> >>>>>> cluster of
> >>>>>>>>>>> 10
> >>>>>>>>>>> nodes. So is there any code or way to measure it?
> >>>>>>>>>>>
> >>>>>>>>>>> I have seen in a cloudera-based presentation that hbase
> >>>> read/write
> >>>>>>>>>>> throughput = millions queries per second.
> >>>>>>>>>>>
> >>>>>>>>>>> So any help please??
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Best Regards,
> >>>>>>>>>>> Dalia Sobhy
> >
> >
> >
> > --
> > Thanks & Regards,
> > Anil Gupta
>

Re: Hbase Read/Write throughput measure

Posted by Dalia Sobhy <da...@hotmail.com>.

Hi anil,

How to recompile it ?

Sent from my iPhone

On 2013-02-03, at 8:00 PM, "anil gupta" <an...@gmail.com> wrote:

> Hi Dalia,
> 
> Recently, i ran into same problem with YCSB. You will need to recompile
> YCSB with HBase0.92. Download the maven project from yscb site and
> recompile it with HBase0.92.
> 
> HTH,
> Anil Gupta
> 
> On Sun, Feb 3, 2013 at 9:49 AM, Dalia Sobhy <da...@hotmail.com>wrote:
> 
>> 
>> Hello Mohamed,
>> 
>> Yes Hbase is running good and already I have a table and and some records
>> there.
>> 
>> I have done what you told me I copied all jar files from lib directory to
>> YSCB/db/hbase/lib/ directory. After restart and writing this command
>> "./bin/ycsb run hbase -p columnfamily=family -P workloads/workloada", I
>> receive the same error.
>> 
>> But I didn't compile, because already I don't have a .java file
>> 
>> ycsb-0.1.4 directory doesn't contain hbase client java file, it contains
>> only this "hbase-binding-0.1.4.jar"
>> 
>> So what to do ?
>> 
>> 
>>> From: dontariq@gmail.com
>>> Date: Sun, 3 Feb 2013 22:54:13 +0530
>>> Subject: Re: Hbase Read/Write throughput measure
>>> To: user@hbase.apache.org
>>> 
>>> Hello Dalia,
>>> 
>>>        Is your Hbase running OK? Copy the hbase jar from your HBASE_HOME
>>> and all the jars from your HBASE_HOME/lib/ directory to your
>>> YSCB/db/hbase/lib/ directory, compile it again and restart everything.
>>> 
>>> Warm Regards,
>>> Tariq
>>> https://mtariq.jux.com/
>>> cloudfront.blogspot.com
>>> 
>>> 
>>> On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
>>> wrote:
>>> 
>>>> 
>>>> Hello Mohamed,
>>>> 
>>>> I believe thats not the case.
>>>> 
>>>> check this blog
>> http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
>>>> 
>>>> It denotes that this error is due to incompatible Hbase client and
>> server
>>>> versions, although I am using Hbase server :0.92 and the server is the
>>>> latest one uploaded by ycsb. So I don't know why this is happening.
>>>> 
>>>> So do you have any idea?
>>>> 
>>>> This is the cmd I am executing:
>>>> 
>>>> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p
>> columnfamily="family"
>>>> -P workloads/workloada
>>>> 
>>>> java -cp
>> /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
>>>> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
>>>> columnfamily=family -P workloads/workloada -t
>>>> YCSB Client 0.1
>>>> Command line: -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family
>> -P
>>>> workloads/workloada -t
>>>> Loading workload...
>>>> Starting test.
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32
>> GMT
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
>> host.name=
>>>> master1.medcloud.com
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:java.version=1.6.0_31
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:java.vendor=Sun Microsystems Inc.
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:java.io.tmpdir=/tmp
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:java.compiler=<NA>
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:os.name
>>>> =Linux
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:os.arch=amd64
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:os.version=3.2.0-23-generic
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
>> user.name
>>>> =root
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:user.home=/root
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:user.dir=/home/dalia/ycsb-0.1.4
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client
>> connection,
>>>> connectString=192.168.2.3:2181 sessionTimeout=180000
>> watcher=hconnection
>>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket connection
>> to
>>>> server /192.168.2.3:2181
>>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection
>> established
>>>> to master1.medcloud.com/192.168.2.3:2181, initiating session
>>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
>>>> complete on server master1.medcloud.com/192.168.2.3:2181, sessionid =
>>>> 0x13c9b45ac70000a, negotiated timeout = 60000
>>>> java.lang.IllegalArgumentException: Not a host:port pair: �
>>>> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
>>>> 
>>>> 
>>>> my hbase-site.xml:
>>>> 
>>>> <configuration>
>>>>  <property>
>>>>    <name>hbase.rootdir</name>
>>>>   <value>hdfs://192.168.2.3:20001/hbase</value>
>>>> 
>>>> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
>>>>    <description>The directory shared by region servers.
>>>>    </description>
>>>>  </property>
>>>>  <property>
>>>>    <name>hbase.master</name>
>>>>        <value>192.168.2.3:60000</value>
>>>>    <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
>>>>    <description>The host and port that the HBase master runs at.
>>>>    </description>
>>>>  </property>
>>>>  <property>
>>>>    <name>hbase.zookeeper.quorum</name>
>>>>    <value>192.168.2.3</value>
>>>>    <!--<value><HBASE_MASTER_SERVER></value>-->
>>>>  </property>
>>>> </configuration>
>>>> 
>>>> So any help please....
>>>> 
>>>> Thanks,
>>>> 
>>>>> From: dontariq@gmail.com
>>>>> Date: Sun, 3 Feb 2013 13:54:28 +0530
>>>>> Subject: Re: Hbase Read/Write throughput measure
>>>>> To: user@hbase.apache.org
>>>>> 
>>>>> Hello Dalia,
>>>>> 
>>>>>       I am sorry for late response, courtesy hectic schedule. Add
>> the
>>>>> hbase-site.xml file to the classpath and see if it works for you.
>>>>> 
>>>>> Warm Regards,
>>>>> Tariq
>>>>> https://mtariq.jux.com/
>>>>> cloudfront.blogspot.com
>>>>> 
>>>>> 
>>>>> On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <
>> dalia.mohsobhy@hotmail.com
>>>>> wrote:
>>>>> 
>>>>>> 
>>>>>> I tried yahoo bench mark I got the following error:
>>>>>> 
>>>>>> java.lang.IllegalArgumentException: Not a host:port
>>>>>> 
>>>>>> So any help,
>>>>>> 
>>>>>> Note that I am using hbase 0.92
>>>>>> 
>>>>>> I have downloaded the lastest ycsb from github
>>>>>> 
>>>>>>> From: dalia.mohsobhy@hotmail.com
>>>>>>> To: user@hbase.apache.org
>>>>>>> Subject: RE: Hbase Read/Write throughput measure
>>>>>>> Date: Sat, 2 Feb 2013 16:47:43 +0200
>>>>>>> 
>>>>>>> 
>>>>>>> any help ??!!!!!!
>>>>>>> 
>>>>>>>> Date: Fri, 1 Feb 2013 17:15:45 -0500
>>>>>>>> Subject: Re: Hbase Read/Write throughput measure
>>>>>>>> From: jean-marc@spaggiari.org
>>>>>>>> To: user@hbase.apache.org
>>>>>>>> 
>>>>>>>> When you will have figure how the command is working, you will
>>>> have to
>>>>>>>> understand the output ;)
>>>>>>>> 
>>>>>>>> This is how I just tried it from my HBase directory:
>>>>>>>> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
>> randomRead
>>>> 10
>>>>>>>> 
>>>>>>>> And this is the output (last lines):
>>>>>>>> 10066326    56314
>>>>>>>> 10171183    56385
>>>>>>>> 10276040    56579
>>>>>>>> 10380897    56553
>>>>>>>> 
>>>>>>>> Does ask me what it means ;) I have, so far, no idea.
>>>>>>>> 
>>>>>>>> JM
>>>>>>>> 
>>>>>>>> 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
>>>>>>>>> Dear Mohamed,
>>>>>>>>> 
>>>>>>>>> I checked this link:
>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
>>>>>>>>> 
>>>>>>>>> But how to use the commands there is no clue, note that I am
>>>> using
>>>>>> cloudera
>>>>>>>>> manager.
>>>>>>>>> 
>>>>>>>>> The other yahoo link I couldn't understand how to use it.
>>>>>>>>> 
>>>>>>>>> So any help please??
>>>>>>>>> 
>>>>>>>>> Sent from my iPad
>>>>>>>>> 
>>>>>>>>> On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <
>> dontariq@gmail.com
>>>>> 
>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hello Dalia,
>>>>>>>>>> 
>>>>>>>>>>         I think the easiest way to measure the read/write
>>>>>> throughput is
>>>>>>>>>> to use "PerformanceEvaluation" tool that comes with the
>> Hbase
>>>>>>>>>> distribution. It spawns a map-reduce job to do the
>> reads/writes
>>>> in
>>>>>>>>>> parallel. Apart from this there are several other ways to
>>>> benchmark
>>>>>> your
>>>>>>>>>> Hbase cluster like YSCB <
>>>> https://github.com/brianfrankcooper/YCSB>.
>>>>>> You
>>>>>>>>>> might find this
>>>>>>>>>> link<
>> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
>>>>>> useful
>>>>>>>>>> that talks about Hbase performance.
>>>>>>>>>> 
>>>>>>>>>> HTH
>>>>>>>>>> 
>>>>>>>>>> Warm Regards,
>>>>>>>>>> Tariq
>>>>>>>>>> https://mtariq.jux.com/
>>>>>>>>>> cloudfront.blogspot.com
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
>>>>>>>>>> <da...@hotmail.com>wrote:
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Dear all,
>>>>>>>>>>> 
>>>>>>>>>>> I want to measure the read/write throughput for a code on a
>>>>>> cluster of
>>>>>>>>>>> 10
>>>>>>>>>>> nodes. So is there any code or way to measure it?
>>>>>>>>>>> 
>>>>>>>>>>> I have seen in a cloudera-based presentation that hbase
>>>> read/write
>>>>>>>>>>> throughput = millions queries per second.
>>>>>>>>>>> 
>>>>>>>>>>> So any help please??
>>>>>>>>>>> 
>>>>>>>>>>> Thanks
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Dalia Sobhy
> 
> 
> 
> -- 
> Thanks & Regards,
> Anil Gupta

Re: Hbase Read/Write throughput measure

Posted by anil gupta <an...@gmail.com>.

Hi Dalia,

Recently, i ran into same problem with YCSB. You will need to recompile
YCSB with HBase0.92. Download the maven project from yscb site and
recompile it with HBase0.92.

HTH,
Anil Gupta

On Sun, Feb 3, 2013 at 9:49 AM, Dalia Sobhy <da...@hotmail.com>wrote:

>
> Hello Mohamed,
>
> Yes Hbase is running good and already I have a table and and some records
> there.
>
> I have done what you told me I copied all jar files from lib directory to
> YSCB/db/hbase/lib/ directory. After restart and writing this command
>  "./bin/ycsb run hbase -p columnfamily=family -P workloads/workloada", I
> receive the same error.
>
> But I didn't compile, because already I don't have a .java file
>
> ycsb-0.1.4 directory doesn't contain hbase client java file, it contains
> only this "hbase-binding-0.1.4.jar"
>
> So what to do ?
>
>
> > From: dontariq@gmail.com
> > Date: Sun, 3 Feb 2013 22:54:13 +0530
> > Subject: Re: Hbase Read/Write throughput measure
> > To: user@hbase.apache.org
> >
> > Hello Dalia,
> >
> >         Is your Hbase running OK? Copy the hbase jar from your HBASE_HOME
> > and all the jars from your HBASE_HOME/lib/ directory to your
> > YSCB/db/hbase/lib/ directory, compile it again and restart everything.
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
> >
> > On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
> >wrote:
> >
> > >
> > > Hello Mohamed,
> > >
> > > I believe thats not the case.
> > >
> > > check this blog
> > >
> http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
> > >
> > > It denotes that this error is due to incompatible Hbase client and
> server
> > > versions, although I am using Hbase server :0.92 and the server is the
> > > latest one uploaded by ycsb. So I don't know why this is happening.
> > >
> > > So do you have any idea?
> > >
> > > This is the cmd I am executing:
> > >
> > > root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p
> columnfamily="family"
> > > -P workloads/workloada
> > >
> > > java -cp
> > >
> /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > > com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
> > > columnfamily=family -P workloads/workloada -t
> > > YCSB Client 0.1
> > > Command line: -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family
> -P
> > > workloads/workloada -t
> > > Loading workload...
> > > Starting test.
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32
> GMT
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> host.name=
> > > master1.medcloud.com
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > environment:java.version=1.6.0_31
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > environment:java.vendor=Sun Microsystems Inc.
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >
> environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >
> environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > environment:java.io.tmpdir=/tmp
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > environment:java.compiler=<NA>
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:os.name
> > > =Linux
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > environment:os.arch=amd64
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > environment:os.version=3.2.0-23-generic
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> user.name
> > > =root
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > environment:user.home=/root
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > environment:user.dir=/home/dalia/ycsb-0.1.4
> > > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client
> connection,
> > > connectString=192.168.2.3:2181 sessionTimeout=180000
> watcher=hconnection
> > > 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket connection
> to
> > > server /192.168.2.3:2181
> > > 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection
> established
> > > to master1.medcloud.com/192.168.2.3:2181, initiating session
> > > 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
> > > complete on server master1.medcloud.com/192.168.2.3:2181, sessionid =
> > > 0x13c9b45ac70000a, negotiated timeout = 60000
> > > java.lang.IllegalArgumentException: Not a host:port pair: �
> > > 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
> > >
> > >
> > > my hbase-site.xml:
> > >
> > > <configuration>
> > >   <property>
> > >     <name>hbase.rootdir</name>
> > >    <value>hdfs://192.168.2.3:20001/hbase</value>
> > >
> > > <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
> > >     <description>The directory shared by region servers.
> > >     </description>
> > >   </property>
> > >   <property>
> > >     <name>hbase.master</name>
> > >         <value>192.168.2.3:60000</value>
> > >     <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
> > >     <description>The host and port that the HBase master runs at.
> > >     </description>
> > >   </property>
> > >   <property>
> > >     <name>hbase.zookeeper.quorum</name>
> > >     <value>192.168.2.3</value>
> > >     <!--<value><HBASE_MASTER_SERVER></value>-->
> > >   </property>
> > > </configuration>
> > >
> > > So any help please....
> > >
> > > Thanks,
> > >
> > > > From: dontariq@gmail.com
> > > > Date: Sun, 3 Feb 2013 13:54:28 +0530
> > > > Subject: Re: Hbase Read/Write throughput measure
> > > > To: user@hbase.apache.org
> > > >
> > > > Hello Dalia,
> > > >
> > > >        I am sorry for late response, courtesy hectic schedule. Add
> the
> > > > hbase-site.xml file to the classpath and see if it works for you.
> > > >
> > > > Warm Regards,
> > > > Tariq
> > > > https://mtariq.jux.com/
> > > > cloudfront.blogspot.com
> > > >
> > > >
> > > > On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <
> dalia.mohsobhy@hotmail.com
> > > >wrote:
> > > >
> > > > >
> > > > > I tried yahoo bench mark I got the following error:
> > > > >
> > > > > java.lang.IllegalArgumentException: Not a host:port
> > > > >
> > > > > So any help,
> > > > >
> > > > > Note that I am using hbase 0.92
> > > > >
> > > > > I have downloaded the lastest ycsb from github
> > > > >
> > > > > > From: dalia.mohsobhy@hotmail.com
> > > > > > To: user@hbase.apache.org
> > > > > > Subject: RE: Hbase Read/Write throughput measure
> > > > > > Date: Sat, 2 Feb 2013 16:47:43 +0200
> > > > > >
> > > > > >
> > > > > > any help ??!!!!!!
> > > > > >
> > > > > > > Date: Fri, 1 Feb 2013 17:15:45 -0500
> > > > > > > Subject: Re: Hbase Read/Write throughput measure
> > > > > > > From: jean-marc@spaggiari.org
> > > > > > > To: user@hbase.apache.org
> > > > > > >
> > > > > > > When you will have figure how the command is working, you will
> > > have to
> > > > > > > understand the output ;)
> > > > > > >
> > > > > > > This is how I just tried it from my HBase directory:
> > > > > > > bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
> randomRead
> > > 10
> > > > > > >
> > > > > > > And this is the output (last lines):
> > > > > > > 10066326    56314
> > > > > > > 10171183    56385
> > > > > > > 10276040    56579
> > > > > > > 10380897    56553
> > > > > > >
> > > > > > > Does ask me what it means ;) I have, so far, no idea.
> > > > > > >
> > > > > > > JM
> > > > > > >
> > > > > > > 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> > > > > > > > Dear Mohamed,
> > > > > > > >
> > > > > > > > I checked this link:
> > > > > > > > http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > > > > > > >
> > > > > > > > But how to use the commands there is no clue, note that I am
> > > using
> > > > > cloudera
> > > > > > > > manager.
> > > > > > > >
> > > > > > > > The other yahoo link I couldn't understand how to use it.
> > > > > > > >
> > > > > > > > So any help please??
> > > > > > > >
> > > > > > > > Sent from my iPad
> > > > > > > >
> > > > > > > > On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <
> dontariq@gmail.com
> > > >
> > > > > wrote:
> > > > > > > >
> > > > > > > >> Hello Dalia,
> > > > > > > >>
> > > > > > > >>          I think the easiest way to measure the read/write
> > > > > throughput is
> > > > > > > >> to use "PerformanceEvaluation" tool that comes with the
> Hbase
> > > > > > > >> distribution. It spawns a map-reduce job to do the
> reads/writes
> > > in
> > > > > > > >> parallel. Apart from this there are several other ways to
> > > benchmark
> > > > > your
> > > > > > > >> Hbase cluster like YSCB <
> > > https://github.com/brianfrankcooper/YCSB>.
> > > > > You
> > > > > > > >> might find this
> > > > > > > >> link<
> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> > > > > useful
> > > > > > > >> that talks about Hbase performance.
> > > > > > > >>
> > > > > > > >> HTH
> > > > > > > >>
> > > > > > > >> Warm Regards,
> > > > > > > >> Tariq
> > > > > > > >> https://mtariq.jux.com/
> > > > > > > >> cloudfront.blogspot.com
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > > > > > > >> <da...@hotmail.com>wrote:
> > > > > > > >>
> > > > > > > >>>
> > > > > > > >>> Dear all,
> > > > > > > >>>
> > > > > > > >>> I want to measure the read/write throughput for a code on a
> > > > > cluster of
> > > > > > > >>> 10
> > > > > > > >>> nodes. So is there any code or way to measure it?
> > > > > > > >>>
> > > > > > > >>> I have seen in a cloudera-based presentation that hbase
> > > read/write
> > > > > > > >>> throughput = millions queries per second.
> > > > > > > >>>
> > > > > > > >>> So any help please??
> > > > > > > >>>
> > > > > > > >>> Thanks
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> Best Regards,
> > > > > > > >>> Dalia Sobhy
> > > > > > > >>>
> > > > > > > >
> > > > > >
> > > > >
> > > > >
> > >
> > >
>
>



-- 
Thanks & Regards,
Anil Gupta

RE: Hbase Read/Write throughput measure

Posted by Dalia Sobhy <da...@hotmail.com>.

Hello Mohamed,

Yes Hbase is running good and already I have a table and and some records there.

I have done what you told me I copied all jar files from lib directory to YSCB/db/hbase/lib/ directory. After restart and writing this command  "./bin/ycsb run hbase -p columnfamily=family -P workloads/workloada", I receive the same error.

But I didn't compile, because already I don't have a .java file

ycsb-0.1.4 directory doesn't contain hbase client java file, it contains only this "hbase-binding-0.1.4.jar"
 
So what to do ?


> From: dontariq@gmail.com
> Date: Sun, 3 Feb 2013 22:54:13 +0530
> Subject: Re: Hbase Read/Write throughput measure
> To: user@hbase.apache.org
> 
> Hello Dalia,
> 
>         Is your Hbase running OK? Copy the hbase jar from your HBASE_HOME
> and all the jars from your HBASE_HOME/lib/ directory to your
> YSCB/db/hbase/lib/ directory, compile it again and restart everything.
> 
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
> 
> 
> On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <da...@hotmail.com>wrote:
> 
> >
> > Hello Mohamed,
> >
> > I believe thats not the case.
> >
> > check this blog
> > http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
> >
> > It denotes that this error is due to incompatible Hbase client and server
> > versions, although I am using Hbase server :0.92 and the server is the
> > latest one uploaded by ycsb. So I don't know why this is happening.
> >
> > So do you have any idea?
> >
> > This is the cmd I am executing:
> >
> > root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p columnfamily="family"
> > -P workloads/workloada
> >
> > java -cp
> > /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
> > columnfamily=family -P workloads/workloada -t
> > YCSB Client 0.1
> > Command line: -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family -P
> > workloads/workloada -t
> > Loading workload...
> > Starting test.
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32 GMT
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:host.name=
> > master1.medcloud.com
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:java.version=1.6.0_31
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:java.vendor=Sun Microsystems Inc.
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:java.io.tmpdir=/tmp
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:java.compiler=<NA>
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:os.name
> > =Linux
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:os.arch=amd64
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:os.version=3.2.0-23-generic
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:user.name
> > =root
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:user.home=/root
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > environment:user.dir=/home/dalia/ycsb-0.1.4
> > 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client connection,
> > connectString=192.168.2.3:2181 sessionTimeout=180000 watcher=hconnection
> > 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket connection to
> > server /192.168.2.3:2181
> > 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection established
> > to master1.medcloud.com/192.168.2.3:2181, initiating session
> > 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
> > complete on server master1.medcloud.com/192.168.2.3:2181, sessionid =
> > 0x13c9b45ac70000a, negotiated timeout = 60000
> > java.lang.IllegalArgumentException: Not a host:port pair: �
> > 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
> >
> >
> > my hbase-site.xml:
> >
> > <configuration>
> >   <property>
> >     <name>hbase.rootdir</name>
> >    <value>hdfs://192.168.2.3:20001/hbase</value>
> >
> > <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
> >     <description>The directory shared by region servers.
> >     </description>
> >   </property>
> >   <property>
> >     <name>hbase.master</name>
> >         <value>192.168.2.3:60000</value>
> >     <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
> >     <description>The host and port that the HBase master runs at.
> >     </description>
> >   </property>
> >   <property>
> >     <name>hbase.zookeeper.quorum</name>
> >     <value>192.168.2.3</value>
> >     <!--<value><HBASE_MASTER_SERVER></value>-->
> >   </property>
> > </configuration>
> >
> > So any help please....
> >
> > Thanks,
> >
> > > From: dontariq@gmail.com
> > > Date: Sun, 3 Feb 2013 13:54:28 +0530
> > > Subject: Re: Hbase Read/Write throughput measure
> > > To: user@hbase.apache.org
> > >
> > > Hello Dalia,
> > >
> > >        I am sorry for late response, courtesy hectic schedule. Add the
> > > hbase-site.xml file to the classpath and see if it works for you.
> > >
> > > Warm Regards,
> > > Tariq
> > > https://mtariq.jux.com/
> > > cloudfront.blogspot.com
> > >
> > >
> > > On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
> > >wrote:
> > >
> > > >
> > > > I tried yahoo bench mark I got the following error:
> > > >
> > > > java.lang.IllegalArgumentException: Not a host:port
> > > >
> > > > So any help,
> > > >
> > > > Note that I am using hbase 0.92
> > > >
> > > > I have downloaded the lastest ycsb from github
> > > >
> > > > > From: dalia.mohsobhy@hotmail.com
> > > > > To: user@hbase.apache.org
> > > > > Subject: RE: Hbase Read/Write throughput measure
> > > > > Date: Sat, 2 Feb 2013 16:47:43 +0200
> > > > >
> > > > >
> > > > > any help ??!!!!!!
> > > > >
> > > > > > Date: Fri, 1 Feb 2013 17:15:45 -0500
> > > > > > Subject: Re: Hbase Read/Write throughput measure
> > > > > > From: jean-marc@spaggiari.org
> > > > > > To: user@hbase.apache.org
> > > > > >
> > > > > > When you will have figure how the command is working, you will
> > have to
> > > > > > understand the output ;)
> > > > > >
> > > > > > This is how I just tried it from my HBase directory:
> > > > > > bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead
> > 10
> > > > > >
> > > > > > And this is the output (last lines):
> > > > > > 10066326    56314
> > > > > > 10171183    56385
> > > > > > 10276040    56579
> > > > > > 10380897    56553
> > > > > >
> > > > > > Does ask me what it means ;) I have, so far, no idea.
> > > > > >
> > > > > > JM
> > > > > >
> > > > > > 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> > > > > > > Dear Mohamed,
> > > > > > >
> > > > > > > I checked this link:
> > > > > > > http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > > > > > >
> > > > > > > But how to use the commands there is no clue, note that I am
> > using
> > > > cloudera
> > > > > > > manager.
> > > > > > >
> > > > > > > The other yahoo link I couldn't understand how to use it.
> > > > > > >
> > > > > > > So any help please??
> > > > > > >
> > > > > > > Sent from my iPad
> > > > > > >
> > > > > > > On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <dontariq@gmail.com
> > >
> > > > wrote:
> > > > > > >
> > > > > > >> Hello Dalia,
> > > > > > >>
> > > > > > >>          I think the easiest way to measure the read/write
> > > > throughput is
> > > > > > >> to use "PerformanceEvaluation" tool that comes with the Hbase
> > > > > > >> distribution. It spawns a map-reduce job to do the reads/writes
> > in
> > > > > > >> parallel. Apart from this there are several other ways to
> > benchmark
> > > > your
> > > > > > >> Hbase cluster like YSCB <
> > https://github.com/brianfrankcooper/YCSB>.
> > > > You
> > > > > > >> might find this
> > > > > > >> link<http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> > > > useful
> > > > > > >> that talks about Hbase performance.
> > > > > > >>
> > > > > > >> HTH
> > > > > > >>
> > > > > > >> Warm Regards,
> > > > > > >> Tariq
> > > > > > >> https://mtariq.jux.com/
> > > > > > >> cloudfront.blogspot.com
> > > > > > >>
> > > > > > >>
> > > > > > >> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > > > > > >> <da...@hotmail.com>wrote:
> > > > > > >>
> > > > > > >>>
> > > > > > >>> Dear all,
> > > > > > >>>
> > > > > > >>> I want to measure the read/write throughput for a code on a
> > > > cluster of
> > > > > > >>> 10
> > > > > > >>> nodes. So is there any code or way to measure it?
> > > > > > >>>
> > > > > > >>> I have seen in a cloudera-based presentation that hbase
> > read/write
> > > > > > >>> throughput = millions queries per second.
> > > > > > >>>
> > > > > > >>> So any help please??
> > > > > > >>>
> > > > > > >>> Thanks
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> Best Regards,
> > > > > > >>> Dalia Sobhy
> > > > > > >>>
> > > > > > >
> > > > >
> > > >
> > > >
> >
> >

Re: Hbase Read/Write throughput measure

Posted by Mohammad Tariq <do...@gmail.com>.

Hello Dalia,

        Is your Hbase running OK? Copy the hbase jar from your HBASE_HOME
and all the jars from your HBASE_HOME/lib/ directory to your
YSCB/db/hbase/lib/ directory, compile it again and restart everything.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <da...@hotmail.com>wrote:

>
> Hello Mohamed,
>
> I believe thats not the case.
>
> check this blog
> http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
>
> It denotes that this error is due to incompatible Hbase client and server
> versions, although I am using Hbase server :0.92 and the server is the
> latest one uploaded by ycsb. So I don't know why this is happening.
>
> So do you have any idea?
>
> This is the cmd I am executing:
>
> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p columnfamily="family"
> -P workloads/workloada
>
> java -cp
> /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
> columnfamily=family -P workloads/workloada -t
> YCSB Client 0.1
> Command line: -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family -P
> workloads/workloada -t
> Loading workload...
> Starting test.
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32 GMT
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:host.name=
> master1.medcloud.com
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.version=1.6.0_31
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.vendor=Sun Microsystems Inc.
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.io.tmpdir=/tmp
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.compiler=<NA>
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:os.name
> =Linux
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:os.arch=amd64
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:os.version=3.2.0-23-generic
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:user.name
> =root
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:user.home=/root
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:user.dir=/home/dalia/ycsb-0.1.4
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client connection,
> connectString=192.168.2.3:2181 sessionTimeout=180000 watcher=hconnection
> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket connection to
> server /192.168.2.3:2181
> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection established
> to master1.medcloud.com/192.168.2.3:2181, initiating session
> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
> complete on server master1.medcloud.com/192.168.2.3:2181, sessionid =
> 0x13c9b45ac70000a, negotiated timeout = 60000
> java.lang.IllegalArgumentException: Not a host:port pair: �
> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
>
>
> my hbase-site.xml:
>
> <configuration>
>   <property>
>     <name>hbase.rootdir</name>
>    <value>hdfs://192.168.2.3:20001/hbase</value>
>
> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
>     <description>The directory shared by region servers.
>     </description>
>   </property>
>   <property>
>     <name>hbase.master</name>
>         <value>192.168.2.3:60000</value>
>     <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
>     <description>The host and port that the HBase master runs at.
>     </description>
>   </property>
>   <property>
>     <name>hbase.zookeeper.quorum</name>
>     <value>192.168.2.3</value>
>     <!--<value><HBASE_MASTER_SERVER></value>-->
>   </property>
> </configuration>
>
> So any help please....
>
> Thanks,
>
> > From: dontariq@gmail.com
> > Date: Sun, 3 Feb 2013 13:54:28 +0530
> > Subject: Re: Hbase Read/Write throughput measure
> > To: user@hbase.apache.org
> >
> > Hello Dalia,
> >
> >        I am sorry for late response, courtesy hectic schedule. Add the
> > hbase-site.xml file to the classpath and see if it works for you.
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
> >
> > On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
> >wrote:
> >
> > >
> > > I tried yahoo bench mark I got the following error:
> > >
> > > java.lang.IllegalArgumentException: Not a host:port
> > >
> > > So any help,
> > >
> > > Note that I am using hbase 0.92
> > >
> > > I have downloaded the lastest ycsb from github
> > >
> > > > From: dalia.mohsobhy@hotmail.com
> > > > To: user@hbase.apache.org
> > > > Subject: RE: Hbase Read/Write throughput measure
> > > > Date: Sat, 2 Feb 2013 16:47:43 +0200
> > > >
> > > >
> > > > any help ??!!!!!!
> > > >
> > > > > Date: Fri, 1 Feb 2013 17:15:45 -0500
> > > > > Subject: Re: Hbase Read/Write throughput measure
> > > > > From: jean-marc@spaggiari.org
> > > > > To: user@hbase.apache.org
> > > > >
> > > > > When you will have figure how the command is working, you will
> have to
> > > > > understand the output ;)
> > > > >
> > > > > This is how I just tried it from my HBase directory:
> > > > > bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead
> 10
> > > > >
> > > > > And this is the output (last lines):
> > > > > 10066326    56314
> > > > > 10171183    56385
> > > > > 10276040    56579
> > > > > 10380897    56553
> > > > >
> > > > > Does ask me what it means ;) I have, so far, no idea.
> > > > >
> > > > > JM
> > > > >
> > > > > 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> > > > > > Dear Mohamed,
> > > > > >
> > > > > > I checked this link:
> > > > > > http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > > > > >
> > > > > > But how to use the commands there is no clue, note that I am
> using
> > > cloudera
> > > > > > manager.
> > > > > >
> > > > > > The other yahoo link I couldn't understand how to use it.
> > > > > >
> > > > > > So any help please??
> > > > > >
> > > > > > Sent from my iPad
> > > > > >
> > > > > > On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <dontariq@gmail.com
> >
> > > wrote:
> > > > > >
> > > > > >> Hello Dalia,
> > > > > >>
> > > > > >>          I think the easiest way to measure the read/write
> > > throughput is
> > > > > >> to use "PerformanceEvaluation" tool that comes with the Hbase
> > > > > >> distribution. It spawns a map-reduce job to do the reads/writes
> in
> > > > > >> parallel. Apart from this there are several other ways to
> benchmark
> > > your
> > > > > >> Hbase cluster like YSCB <
> https://github.com/brianfrankcooper/YCSB>.
> > > You
> > > > > >> might find this
> > > > > >> link<http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> > > useful
> > > > > >> that talks about Hbase performance.
> > > > > >>
> > > > > >> HTH
> > > > > >>
> > > > > >> Warm Regards,
> > > > > >> Tariq
> > > > > >> https://mtariq.jux.com/
> > > > > >> cloudfront.blogspot.com
> > > > > >>
> > > > > >>
> > > > > >> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > > > > >> <da...@hotmail.com>wrote:
> > > > > >>
> > > > > >>>
> > > > > >>> Dear all,
> > > > > >>>
> > > > > >>> I want to measure the read/write throughput for a code on a
> > > cluster of
> > > > > >>> 10
> > > > > >>> nodes. So is there any code or way to measure it?
> > > > > >>>
> > > > > >>> I have seen in a cloudera-based presentation that hbase
> read/write
> > > > > >>> throughput = millions queries per second.
> > > > > >>>
> > > > > >>> So any help please??
> > > > > >>>
> > > > > >>> Thanks
> > > > > >>>
> > > > > >>>
> > > > > >>> Best Regards,
> > > > > >>> Dalia Sobhy
> > > > > >>>
> > > > > >
> > > >
> > >
> > >
>
>

Re: Hbase Read/Write throughput measure

Posted by anil gupta <an...@gmail.com>.

Hi Dalia,

Recently, i ran into the same problem with yscb. You will have to recompile
YCSB by yourself by downloading the maven project. The problem is that YCSB
is not compiled with 0.92 even though they claim to do so. So, once you
locally compile YCSB with HBase0.92 then you should be good to go.

HTH,
Anil Gupta

On Sun, Feb 3, 2013 at 8:48 AM, Dalia Sobhy <da...@hotmail.com>wrote:

>
> Hello Mohamed,
>
> I believe thats not the case.
>
> check this blog
> http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
>
> It denotes that this error is due to incompatible Hbase client and server
> versions, although I am using Hbase server :0.92 and the server is the
> latest one uploaded by ycsb. So I don't know why this is happening.
>
> So do you have any idea?
>
> This is the cmd I am executing:
>
> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p columnfamily="family"
> -P workloads/workloada
>
> java -cp
> /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
> columnfamily=family -P workloads/workloada -t
> YCSB Client 0.1
> Command line: -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family -P
> workloads/workloada -t
> Loading workload...
> Starting test.
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32 GMT
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:host.name=
> master1.medcloud.com
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.version=1.6.0_31
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.vendor=Sun Microsystems Inc.
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.io.tmpdir=/tmp
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.compiler=<NA>
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:os.name
> =Linux
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:os.arch=amd64
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:os.version=3.2.0-23-generic
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:user.name
> =root
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:user.home=/root
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:user.dir=/home/dalia/ycsb-0.1.4
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client connection,
> connectString=192.168.2.3:2181 sessionTimeout=180000 watcher=hconnection
> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket connection to
> server /192.168.2.3:2181
> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection established
> to master1.medcloud.com/192.168.2.3:2181, initiating session
> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
> complete on server master1.medcloud.com/192.168.2.3:2181, sessionid =
> 0x13c9b45ac70000a, negotiated timeout = 60000
> java.lang.IllegalArgumentException: Not a host:port pair: �
> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
>
>
> my hbase-site.xml:
>
> <configuration>
>   <property>
>     <name>hbase.rootdir</name>
>    <value>hdfs://192.168.2.3:20001/hbase</value>
>
> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
>     <description>The directory shared by region servers.
>     </description>
>   </property>
>   <property>
>     <name>hbase.master</name>
>         <value>192.168.2.3:60000</value>
>     <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
>     <description>The host and port that the HBase master runs at.
>     </description>
>   </property>
>   <property>
>     <name>hbase.zookeeper.quorum</name>
>     <value>192.168.2.3</value>
>     <!--<value><HBASE_MASTER_SERVER></value>-->
>   </property>
> </configuration>
>
> So any help please....
>
> Thanks,
>
> > From: dontariq@gmail.com
> > Date: Sun, 3 Feb 2013 13:54:28 +0530
> > Subject: Re: Hbase Read/Write throughput measure
> > To: user@hbase.apache.org
> >
> > Hello Dalia,
> >
> >        I am sorry for late response, courtesy hectic schedule. Add the
> > hbase-site.xml file to the classpath and see if it works for you.
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
> >
> > On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
> >wrote:
> >
> > >
> > > I tried yahoo bench mark I got the following error:
> > >
> > > java.lang.IllegalArgumentException: Not a host:port
> > >
> > > So any help,
> > >
> > > Note that I am using hbase 0.92
> > >
> > > I have downloaded the lastest ycsb from github
> > >
> > > > From: dalia.mohsobhy@hotmail.com
> > > > To: user@hbase.apache.org
> > > > Subject: RE: Hbase Read/Write throughput measure
> > > > Date: Sat, 2 Feb 2013 16:47:43 +0200
> > > >
> > > >
> > > > any help ??!!!!!!
> > > >
> > > > > Date: Fri, 1 Feb 2013 17:15:45 -0500
> > > > > Subject: Re: Hbase Read/Write throughput measure
> > > > > From: jean-marc@spaggiari.org
> > > > > To: user@hbase.apache.org
> > > > >
> > > > > When you will have figure how the command is working, you will
> have to
> > > > > understand the output ;)
> > > > >
> > > > > This is how I just tried it from my HBase directory:
> > > > > bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead
> 10
> > > > >
> > > > > And this is the output (last lines):
> > > > > 10066326    56314
> > > > > 10171183    56385
> > > > > 10276040    56579
> > > > > 10380897    56553
> > > > >
> > > > > Does ask me what it means ;) I have, so far, no idea.
> > > > >
> > > > > JM
> > > > >
> > > > > 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> > > > > > Dear Mohamed,
> > > > > >
> > > > > > I checked this link:
> > > > > > http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > > > > >
> > > > > > But how to use the commands there is no clue, note that I am
> using
> > > cloudera
> > > > > > manager.
> > > > > >
> > > > > > The other yahoo link I couldn't understand how to use it.
> > > > > >
> > > > > > So any help please??
> > > > > >
> > > > > > Sent from my iPad
> > > > > >
> > > > > > On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <dontariq@gmail.com
> >
> > > wrote:
> > > > > >
> > > > > >> Hello Dalia,
> > > > > >>
> > > > > >>          I think the easiest way to measure the read/write
> > > throughput is
> > > > > >> to use "PerformanceEvaluation" tool that comes with the Hbase
> > > > > >> distribution. It spawns a map-reduce job to do the reads/writes
> in
> > > > > >> parallel. Apart from this there are several other ways to
> benchmark
> > > your
> > > > > >> Hbase cluster like YSCB <
> https://github.com/brianfrankcooper/YCSB>.
> > > You
> > > > > >> might find this
> > > > > >> link<http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> > > useful
> > > > > >> that talks about Hbase performance.
> > > > > >>
> > > > > >> HTH
> > > > > >>
> > > > > >> Warm Regards,
> > > > > >> Tariq
> > > > > >> https://mtariq.jux.com/
> > > > > >> cloudfront.blogspot.com
> > > > > >>
> > > > > >>
> > > > > >> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > > > > >> <da...@hotmail.com>wrote:
> > > > > >>
> > > > > >>>
> > > > > >>> Dear all,
> > > > > >>>
> > > > > >>> I want to measure the read/write throughput for a code on a
> > > cluster of
> > > > > >>> 10
> > > > > >>> nodes. So is there any code or way to measure it?
> > > > > >>>
> > > > > >>> I have seen in a cloudera-based presentation that hbase
> read/write
> > > > > >>> throughput = millions queries per second.
> > > > > >>>
> > > > > >>> So any help please??
> > > > > >>>
> > > > > >>> Thanks
> > > > > >>>
> > > > > >>>
> > > > > >>> Best Regards,
> > > > > >>> Dalia Sobhy
> > > > > >>>
> > > > > >
> > > >
> > >
> > >
>
>



-- 
Thanks & Regards,
Anil Gupta

RE: Hbase Read/Write throughput measure

Posted by Dalia Sobhy <da...@hotmail.com>.

Hello Mohamed,

I believe thats not the case.

check this blog
http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair

It denotes that this error is due to incompatible Hbase client and server versions, although I am using Hbase server :0.92 and the server is the latest one uploaded by ycsb. So I don't know why this is happening.

So do you have any idea?

This is the cmd I am executing:

root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p columnfamily="family" -P workloads/workloada

java -cp /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family -P workloads/workloada -t
YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family -P workloads/workloada -t
Loading workload...
Starting test.
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32 GMT
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:host.name=master1.medcloud.com
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_31
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc.
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:os.version=3.2.0-23-generic
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:user.name=root
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:user.home=/root
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/dalia/ycsb-0.1.4
13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=192.168.2.3:2181 sessionTimeout=180000 watcher=hconnection
13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket connection to server /192.168.2.3:2181
13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection established to master1.medcloud.com/192.168.2.3:2181, initiating session
13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment complete on server master1.medcloud.com/192.168.2.3:2181, sessionid = 0x13c9b45ac70000a, negotiated timeout = 60000
java.lang.IllegalArgumentException: Not a host:port pair: �1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801


my hbase-site.xml:

<configuration>
  <property>
    <name>hbase.rootdir</name>
   <value>hdfs://192.168.2.3:20001/hbase</value>

<!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
    <description>The directory shared by region servers.
    </description>
  </property>
  <property>
    <name>hbase.master</name>
        <value>192.168.2.3:60000</value>
    <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
    <description>The host and port that the HBase master runs at.
    </description>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>192.168.2.3</value>
    <!--<value><HBASE_MASTER_SERVER></value>-->
  </property>
</configuration>

So any help please....

Thanks,

> From: dontariq@gmail.com
> Date: Sun, 3 Feb 2013 13:54:28 +0530
> Subject: Re: Hbase Read/Write throughput measure
> To: user@hbase.apache.org
> 
> Hello Dalia,
> 
>        I am sorry for late response, courtesy hectic schedule. Add the
> hbase-site.xml file to the classpath and see if it works for you.
> 
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
> 
> 
> On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <da...@hotmail.com>wrote:
> 
> >
> > I tried yahoo bench mark I got the following error:
> >
> > java.lang.IllegalArgumentException: Not a host:port
> >
> > So any help,
> >
> > Note that I am using hbase 0.92
> >
> > I have downloaded the lastest ycsb from github
> >
> > > From: dalia.mohsobhy@hotmail.com
> > > To: user@hbase.apache.org
> > > Subject: RE: Hbase Read/Write throughput measure
> > > Date: Sat, 2 Feb 2013 16:47:43 +0200
> > >
> > >
> > > any help ??!!!!!!
> > >
> > > > Date: Fri, 1 Feb 2013 17:15:45 -0500
> > > > Subject: Re: Hbase Read/Write throughput measure
> > > > From: jean-marc@spaggiari.org
> > > > To: user@hbase.apache.org
> > > >
> > > > When you will have figure how the command is working, you will have to
> > > > understand the output ;)
> > > >
> > > > This is how I just tried it from my HBase directory:
> > > > bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 10
> > > >
> > > > And this is the output (last lines):
> > > > 10066326    56314
> > > > 10171183    56385
> > > > 10276040    56579
> > > > 10380897    56553
> > > >
> > > > Does ask me what it means ;) I have, so far, no idea.
> > > >
> > > > JM
> > > >
> > > > 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> > > > > Dear Mohamed,
> > > > >
> > > > > I checked this link:
> > > > > http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > > > >
> > > > > But how to use the commands there is no clue, note that I am using
> > cloudera
> > > > > manager.
> > > > >
> > > > > The other yahoo link I couldn't understand how to use it.
> > > > >
> > > > > So any help please??
> > > > >
> > > > > Sent from my iPad
> > > > >
> > > > > On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <do...@gmail.com>
> > wrote:
> > > > >
> > > > >> Hello Dalia,
> > > > >>
> > > > >>          I think the easiest way to measure the read/write
> > throughput is
> > > > >> to use "PerformanceEvaluation" tool that comes with the Hbase
> > > > >> distribution. It spawns a map-reduce job to do the reads/writes in
> > > > >> parallel. Apart from this there are several other ways to benchmark
> > your
> > > > >> Hbase cluster like YSCB <https://github.com/brianfrankcooper/YCSB>.
> > You
> > > > >> might find this
> > > > >> link<http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> > useful
> > > > >> that talks about Hbase performance.
> > > > >>
> > > > >> HTH
> > > > >>
> > > > >> Warm Regards,
> > > > >> Tariq
> > > > >> https://mtariq.jux.com/
> > > > >> cloudfront.blogspot.com
> > > > >>
> > > > >>
> > > > >> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > > > >> <da...@hotmail.com>wrote:
> > > > >>
> > > > >>>
> > > > >>> Dear all,
> > > > >>>
> > > > >>> I want to measure the read/write throughput for a code on a
> > cluster of
> > > > >>> 10
> > > > >>> nodes. So is there any code or way to measure it?
> > > > >>>
> > > > >>> I have seen in a cloudera-based presentation that hbase read/write
> > > > >>> throughput = millions queries per second.
> > > > >>>
> > > > >>> So any help please??
> > > > >>>
> > > > >>> Thanks
> > > > >>>
> > > > >>>
> > > > >>> Best Regards,
> > > > >>> Dalia Sobhy
> > > > >>>
> > > > >
> > >
> >
> >

Re: Hbase Read/Write throughput measure

Posted by Mohammad Tariq <do...@gmail.com>.

Hello Dalia,

       I am sorry for late response, courtesy hectic schedule. Add the
hbase-site.xml file to the classpath and see if it works for you.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <da...@hotmail.com>wrote:

>
> I tried yahoo bench mark I got the following error:
>
> java.lang.IllegalArgumentException: Not a host:port
>
> So any help,
>
> Note that I am using hbase 0.92
>
> I have downloaded the lastest ycsb from github
>
> > From: dalia.mohsobhy@hotmail.com
> > To: user@hbase.apache.org
> > Subject: RE: Hbase Read/Write throughput measure
> > Date: Sat, 2 Feb 2013 16:47:43 +0200
> >
> >
> > any help ??!!!!!!
> >
> > > Date: Fri, 1 Feb 2013 17:15:45 -0500
> > > Subject: Re: Hbase Read/Write throughput measure
> > > From: jean-marc@spaggiari.org
> > > To: user@hbase.apache.org
> > >
> > > When you will have figure how the command is working, you will have to
> > > understand the output ;)
> > >
> > > This is how I just tried it from my HBase directory:
> > > bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 10
> > >
> > > And this is the output (last lines):
> > > 10066326    56314
> > > 10171183    56385
> > > 10276040    56579
> > > 10380897    56553
> > >
> > > Does ask me what it means ;) I have, so far, no idea.
> > >
> > > JM
> > >
> > > 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> > > > Dear Mohamed,
> > > >
> > > > I checked this link:
> > > > http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > > >
> > > > But how to use the commands there is no clue, note that I am using
> cloudera
> > > > manager.
> > > >
> > > > The other yahoo link I couldn't understand how to use it.
> > > >
> > > > So any help please??
> > > >
> > > > Sent from my iPad
> > > >
> > > > On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <do...@gmail.com>
> wrote:
> > > >
> > > >> Hello Dalia,
> > > >>
> > > >>          I think the easiest way to measure the read/write
> throughput is
> > > >> to use "PerformanceEvaluation" tool that comes with the Hbase
> > > >> distribution. It spawns a map-reduce job to do the reads/writes in
> > > >> parallel. Apart from this there are several other ways to benchmark
> your
> > > >> Hbase cluster like YSCB <https://github.com/brianfrankcooper/YCSB>.
> You
> > > >> might find this
> > > >> link<http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> useful
> > > >> that talks about Hbase performance.
> > > >>
> > > >> HTH
> > > >>
> > > >> Warm Regards,
> > > >> Tariq
> > > >> https://mtariq.jux.com/
> > > >> cloudfront.blogspot.com
> > > >>
> > > >>
> > > >> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > > >> <da...@hotmail.com>wrote:
> > > >>
> > > >>>
> > > >>> Dear all,
> > > >>>
> > > >>> I want to measure the read/write throughput for a code on a
> cluster of
> > > >>> 10
> > > >>> nodes. So is there any code or way to measure it?
> > > >>>
> > > >>> I have seen in a cloudera-based presentation that hbase read/write
> > > >>> throughput = millions queries per second.
> > > >>>
> > > >>> So any help please??
> > > >>>
> > > >>> Thanks
> > > >>>
> > > >>>
> > > >>> Best Regards,
> > > >>> Dalia Sobhy
> > > >>>
> > > >
> >
>
>

RE: Hbase Read/Write throughput measure

Posted by Dalia Sobhy <da...@hotmail.com>.

I tried yahoo bench mark I got the following error:

java.lang.IllegalArgumentException: Not a host:port 

So any help,

Note that I am using hbase 0.92

I have downloaded the lastest ycsb from github

> From: dalia.mohsobhy@hotmail.com
> To: user@hbase.apache.org
> Subject: RE: Hbase Read/Write throughput measure
> Date: Sat, 2 Feb 2013 16:47:43 +0200
> 
> 
> any help ??!!!!!!
> 
> > Date: Fri, 1 Feb 2013 17:15:45 -0500
> > Subject: Re: Hbase Read/Write throughput measure
> > From: jean-marc@spaggiari.org
> > To: user@hbase.apache.org
> > 
> > When you will have figure how the command is working, you will have to
> > understand the output ;)
> > 
> > This is how I just tried it from my HBase directory:
> > bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 10
> > 
> > And this is the output (last lines):
> > 10066326	56314
> > 10171183	56385
> > 10276040	56579
> > 10380897	56553
> > 
> > Does ask me what it means ;) I have, so far, no idea.
> > 
> > JM
> > 
> > 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> > > Dear Mohamed,
> > >
> > > I checked this link:
> > > http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > >
> > > But how to use the commands there is no clue, note that I am using cloudera
> > > manager.
> > >
> > > The other yahoo link I couldn't understand how to use it.
> > >
> > > So any help please??
> > >
> > > Sent from my iPad
> > >
> > > On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <do...@gmail.com> wrote:
> > >
> > >> Hello Dalia,
> > >>
> > >>          I think the easiest way to measure the read/write throughput is
> > >> to use "PerformanceEvaluation" tool that comes with the Hbase
> > >> distribution. It spawns a map-reduce job to do the reads/writes in
> > >> parallel. Apart from this there are several other ways to benchmark your
> > >> Hbase cluster like YSCB <https://github.com/brianfrankcooper/YCSB>. You
> > >> might find this
> > >> link<http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation> useful
> > >> that talks about Hbase performance.
> > >>
> > >> HTH
> > >>
> > >> Warm Regards,
> > >> Tariq
> > >> https://mtariq.jux.com/
> > >> cloudfront.blogspot.com
> > >>
> > >>
> > >> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > >> <da...@hotmail.com>wrote:
> > >>
> > >>>
> > >>> Dear all,
> > >>>
> > >>> I want to measure the read/write throughput for a code on a cluster of
> > >>> 10
> > >>> nodes. So is there any code or way to measure it?
> > >>>
> > >>> I have seen in a cloudera-based presentation that hbase read/write
> > >>> throughput = millions queries per second.
> > >>>
> > >>> So any help please??
> > >>>
> > >>> Thanks
> > >>>
> > >>>
> > >>> Best Regards,
> > >>> Dalia Sobhy
> > >>>
> > >
>

RE: Hbase Read/Write throughput measure

Posted by Dalia Sobhy <da...@hotmail.com>.

any help ??!!!!!!

> Date: Fri, 1 Feb 2013 17:15:45 -0500
> Subject: Re: Hbase Read/Write throughput measure
> From: jean-marc@spaggiari.org
> To: user@hbase.apache.org
> 
> When you will have figure how the command is working, you will have to
> understand the output ;)
> 
> This is how I just tried it from my HBase directory:
> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 10
> 
> And this is the output (last lines):
> 10066326	56314
> 10171183	56385
> 10276040	56579
> 10380897	56553
> 
> Does ask me what it means ;) I have, so far, no idea.
> 
> JM
> 
> 2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> > Dear Mohamed,
> >
> > I checked this link:
> > http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> >
> > But how to use the commands there is no clue, note that I am using cloudera
> > manager.
> >
> > The other yahoo link I couldn't understand how to use it.
> >
> > So any help please??
> >
> > Sent from my iPad
> >
> > On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <do...@gmail.com> wrote:
> >
> >> Hello Dalia,
> >>
> >>          I think the easiest way to measure the read/write throughput is
> >> to use "PerformanceEvaluation" tool that comes with the Hbase
> >> distribution. It spawns a map-reduce job to do the reads/writes in
> >> parallel. Apart from this there are several other ways to benchmark your
> >> Hbase cluster like YSCB <https://github.com/brianfrankcooper/YCSB>. You
> >> might find this
> >> link<http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation> useful
> >> that talks about Hbase performance.
> >>
> >> HTH
> >>
> >> Warm Regards,
> >> Tariq
> >> https://mtariq.jux.com/
> >> cloudfront.blogspot.com
> >>
> >>
> >> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> >> <da...@hotmail.com>wrote:
> >>
> >>>
> >>> Dear all,
> >>>
> >>> I want to measure the read/write throughput for a code on a cluster of
> >>> 10
> >>> nodes. So is there any code or way to measure it?
> >>>
> >>> I have seen in a cloudera-based presentation that hbase read/write
> >>> throughput = millions queries per second.
> >>>
> >>> So any help please??
> >>>
> >>> Thanks
> >>>
> >>>
> >>> Best Regards,
> >>> Dalia Sobhy
> >>>
> >

Re: Hbase Read/Write throughput measure

Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.

When you will have figure how the command is working, you will have to
understand the output ;)

This is how I just tried it from my HBase directory:
bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 10

And this is the output (last lines):
10066326	56314
10171183	56385
10276040	56579
10380897	56553

Does ask me what it means ;) I have, so far, no idea.

JM

2013/2/1, Dalia Sobhy <da...@hotmail.com>:
> Dear Mohamed,
>
> I checked this link:
> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
>
> But how to use the commands there is no clue, note that I am using cloudera
> manager.
>
> The other yahoo link I couldn't understand how to use it.
>
> So any help please??
>
> Sent from my iPad
>
> On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <do...@gmail.com> wrote:
>
>> Hello Dalia,
>>
>>          I think the easiest way to measure the read/write throughput is
>> to use "PerformanceEvaluation" tool that comes with the Hbase
>> distribution. It spawns a map-reduce job to do the reads/writes in
>> parallel. Apart from this there are several other ways to benchmark your
>> Hbase cluster like YSCB <https://github.com/brianfrankcooper/YCSB>. You
>> might find this
>> link<http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation> useful
>> that talks about Hbase performance.
>>
>> HTH
>>
>> Warm Regards,
>> Tariq
>> https://mtariq.jux.com/
>> cloudfront.blogspot.com
>>
>>
>> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
>> <da...@hotmail.com>wrote:
>>
>>>
>>> Dear all,
>>>
>>> I want to measure the read/write throughput for a code on a cluster of
>>> 10
>>> nodes. So is there any code or way to measure it?
>>>
>>> I have seen in a cloudera-based presentation that hbase read/write
>>> throughput = millions queries per second.
>>>
>>> So any help please??
>>>
>>> Thanks
>>>
>>>
>>> Best Regards,
>>> Dalia Sobhy
>>>
>

Re: Hbase Read/Write throughput measure

Posted by Dalia Sobhy <da...@hotmail.com>.

Dear Mohamed,

I checked this link: http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation

But how to use the commands there is no clue, note that I am using cloudera manager.

The other yahoo link I couldn't understand how to use it.

So any help please??

Sent from my iPad

On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <do...@gmail.com> wrote:

> Hello Dalia,
> 
>          I think the easiest way to measure the read/write throughput is
> to use "PerformanceEvaluation" tool that comes with the Hbase
> distribution. It spawns a map-reduce job to do the reads/writes in
> parallel. Apart from this there are several other ways to benchmark your
> Hbase cluster like YSCB <https://github.com/brianfrankcooper/YCSB>. You
> might find this
> link<http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation> useful
> that talks about Hbase performance.
> 
> HTH
> 
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
> 
> 
> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy <da...@hotmail.com>wrote:
> 
>> 
>> Dear all,
>> 
>> I want to measure the read/write throughput for a code on a cluster of 10
>> nodes. So is there any code or way to measure it?
>> 
>> I have seen in a cloudera-based presentation that hbase read/write
>> throughput = millions queries per second.
>> 
>> So any help please??
>> 
>> Thanks
>> 
>> 
>> Best Regards,
>> Dalia Sobhy
>>

Re: Hbase Read/Write throughput measure

Posted by Mohammad Tariq <do...@gmail.com>.

Hello Dalia,

          I think the easiest way to measure the read/write throughput is
to use "PerformanceEvaluation" tool that comes with the Hbase
distribution. It spawns a map-reduce job to do the reads/writes in
parallel. Apart from this there are several other ways to benchmark your
Hbase cluster like YSCB <https://github.com/brianfrankcooper/YCSB>. You
might find this
link<http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation> useful
that talks about Hbase performance.

HTH

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy <da...@hotmail.com>wrote:

>
> Dear all,
>
> I want to measure the read/write throughput for a code on a cluster of 10
> nodes. So is there any code or way to measure it?
>
> I have seen in a cloudera-based presentation that hbase read/write
> throughput = millions queries per second.
>
> So any help please??
>
> Thanks
>
>
> Best Regards,
> Dalia Sobhy
>