You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Tao Xie <xi...@gmail.com> on 2010/09/25 17:19:27 UTC
anybody running ycsb?
I want to reproduce the results in the ycsb paper. I run hbase 0.20.6 and
hadoop 0.20.2. My cluster is like this:
1 Node as HMaster + ZK
6 Nodes as DN, RS
1 Node as Hbase client.
I think this environment is something like the one used by the paper.
When I run tests like workloadb with 100 threads, I get at most 2500 ops/sec
throughput and read latency is about 40~50 ms, which is much higher than the
paper results (about 10+ ms). I wonder if anybody is running ycsb too and
can give me some hints.
Thanks in advance.
Re: anybody running ycsb?
Posted by Alexey Kovyrin <al...@kovyrin.net>.
AFAIR raid10 should be around 2x faster on both read and write operations comparing to a simple jbod.
Sent from my iPad
On 2010-09-26, at 10:09 PM, Tao Xie <xi...@gmail.com> wrote:
> Thanks, Ryan. But in the paper they said their Read operations retrieves an
> entire record. And my nodes get even more memory than theirs. They has only
> 8GB ram while I got 24GB. The only difference is I use JBOD rather than
> RAID10 array. I think JBOD would be better for HDFS than RAID. Is that
> right?
>
>
> 2010/9/27 Ryan Rawson <ry...@gmail.com>
>
>> hey,
>>
>> there are a lot of settings which all affect speed, you might want to
>> make sure you are running the exact config the paper is. for example
>> "readallfields".. if you have a wide column setting it to 'true' will
>> go much slower (since you are returning a lot more data) than if you
>> set it to false. Also there are known bottlenecks in our HTable
>> client, which exist to reduce the # of socket connections, but you can
>> get higher performance with multiple VMs, although that is more
>> complex.
>>
>> Also dont forget hardware.. .make sure your HW is the same as the papers.
>>
>> Good luck!
>> -ryan
>>
>> On Sat, Sep 25, 2010 at 8:19 AM, Tao Xie <xi...@gmail.com> wrote:
>>> I want to reproduce the results in the ycsb paper. I run hbase 0.20.6
>> and
>>> hadoop 0.20.2. My cluster is like this:
>>>
>>> 1 Node as HMaster + ZK
>>>
>>> 6 Nodes as DN, RS
>>>
>>> 1 Node as Hbase client.
>>>
>>> I think this environment is something like the one used by the paper.
>>>
>>> When I run tests like workloadb with 100 threads, I get at most 2500
>> ops/sec
>>> throughput and read latency is about 40~50 ms, which is much higher than
>> the
>>> paper results (about 10+ ms). I wonder if anybody is running ycsb too and
>>> can give me some hints.
>>>
>>>
>>>
>>> Thanks in advance.
>>>
>>
Re: anybody running ycsb?
Posted by Tao Xie <xi...@gmail.com>.
Thanks, Ryan. But in the paper they said their Read operations retrieves an
entire record. And my nodes get even more memory than theirs. They has only
8GB ram while I got 24GB. The only difference is I use JBOD rather than
RAID10 array. I think JBOD would be better for HDFS than RAID. Is that
right?
2010/9/27 Ryan Rawson <ry...@gmail.com>
> hey,
>
> there are a lot of settings which all affect speed, you might want to
> make sure you are running the exact config the paper is. for example
> "readallfields".. if you have a wide column setting it to 'true' will
> go much slower (since you are returning a lot more data) than if you
> set it to false. Also there are known bottlenecks in our HTable
> client, which exist to reduce the # of socket connections, but you can
> get higher performance with multiple VMs, although that is more
> complex.
>
> Also dont forget hardware.. .make sure your HW is the same as the papers.
>
> Good luck!
> -ryan
>
> On Sat, Sep 25, 2010 at 8:19 AM, Tao Xie <xi...@gmail.com> wrote:
> > I want to reproduce the results in the ycsb paper. I run hbase 0.20.6
> and
> > hadoop 0.20.2. My cluster is like this:
> >
> > 1 Node as HMaster + ZK
> >
> > 6 Nodes as DN, RS
> >
> > 1 Node as Hbase client.
> >
> > I think this environment is something like the one used by the paper.
> >
> > When I run tests like workloadb with 100 threads, I get at most 2500
> ops/sec
> > throughput and read latency is about 40~50 ms, which is much higher than
> the
> > paper results (about 10+ ms). I wonder if anybody is running ycsb too and
> > can give me some hints.
> >
> >
> >
> > Thanks in advance.
> >
>
Re: anybody running ycsb?
Posted by Ryan Rawson <ry...@gmail.com>.
hey,
there are a lot of settings which all affect speed, you might want to
make sure you are running the exact config the paper is. for example
"readallfields".. if you have a wide column setting it to 'true' will
go much slower (since you are returning a lot more data) than if you
set it to false. Also there are known bottlenecks in our HTable
client, which exist to reduce the # of socket connections, but you can
get higher performance with multiple VMs, although that is more
complex.
Also dont forget hardware.. .make sure your HW is the same as the papers.
Good luck!
-ryan
On Sat, Sep 25, 2010 at 8:19 AM, Tao Xie <xi...@gmail.com> wrote:
> I want to reproduce the results in the ycsb paper. I run hbase 0.20.6 and
> hadoop 0.20.2. My cluster is like this:
>
> 1 Node as HMaster + ZK
>
> 6 Nodes as DN, RS
>
> 1 Node as Hbase client.
>
> I think this environment is something like the one used by the paper.
>
> When I run tests like workloadb with 100 threads, I get at most 2500 ops/sec
> throughput and read latency is about 40~50 ms, which is much higher than the
> paper results (about 10+ ms). I wonder if anybody is running ycsb too and
> can give me some hints.
>
>
>
> Thanks in advance.
>