You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Charles Kaminski <fr...@yahoo.com> on 2008/02/08 01:03:27 UTC
Evaluating HBase 3
Hi All,
We're running into sever performance issues. I'm
hoping that there is something simple we can do to
resolve the issues. Any help would be appreciated.
Here's what we did:
1. Loaded 1,000 records into a table with only two
columns - row and content:. Row data is 12 bytes and
content: data is 23 bytes long.
2. Using HBase, selected a single record based on row
in the where clause. Did this for a few different
records. Performance was consistantly 0.01 seconds as
reported by Hbase.
3. Loaded 1,000,000 records into the same table. This
took 248 seconds using random row values.
4. Ran the exact same select statments again as in
step 2. These consistantly took 2 to 3 seconds to
return a single record.
2 to 3 seconds to return a single record using a key
value suggests a major issue with our setup. I'm
hoping you agree and can point us to something we're
doing wrong.
____________________________________________________________________________________
Looking for last minute shopping deals?
Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
Re: Evaluating HBase 3
Posted by Bryan Duxbury <br...@rapleaf.com>.
You say "selected a single record based on row in where clause". Are
you working in the shell?
-Bryan
On Feb 7, 2008, at 4:03 PM, Charles Kaminski wrote:
> Hi All,
>
> We're running into sever performance issues. I'm
> hoping that there is something simple we can do to
> resolve the issues. Any help would be appreciated.
>
> Here's what we did:
> 1. Loaded 1,000 records into a table with only two
> columns - row and content:. Row data is 12 bytes and
> content: data is 23 bytes long.
> 2. Using HBase, selected a single record based on row
> in the where clause. Did this for a few different
> records. Performance was consistantly 0.01 seconds as
> reported by Hbase.
> 3. Loaded 1,000,000 records into the same table. This
> took 248 seconds using random row values.
> 4. Ran the exact same select statments again as in
> step 2. These consistantly took 2 to 3 seconds to
> return a single record.
>
> 2 to 3 seconds to return a single record using a key
> value suggests a major issue with our setup. I'm
> hoping you agree and can point us to something we're
> doing wrong.
>
>
>
>
>
>
>
> ______________________________________________________________________
> ______________
> Looking for last minute shopping deals?
> Find them fast with Yahoo! Search. http://tools.search.yahoo.com/
> newsearch/category.php?category=shopping
Re: Evaluating HBase 3
Posted by Charles Kaminski <fr...@yahoo.com>.
St.Ack and Bryan,
Turns out it was inconsistant testing on our part.
When we tested with HBase Shell on the server and got
similar results, we thought we were ruling out any
issues with machines connecteding to the cluster.
The posts questioning HBase Shell as a good test
prompted us to go back and take a more indepth review.
Thanks again!
--- stack <st...@duboce.net> wrote:
> Lets try and figure out whats going on Charles.
>
> The figures on the end of this page have us random
> reading bigger values
> out of a table of 1M rows at somewhere between 150
> and 300 rows a
> second, dependent on hbase version (Whats your
> version?)
>
> Want to send us the code your java apps are using to
> access hbase so we
> can check it out?
>
> Thanks,
> St.Ack
>
>
> Charles Kaminski wrote:
> > Hi St.Ack,
> >
> > Thanks for the response. The performance changes
> > below are consistent with what we find in our java
> > app. We used Hbase Shell directly on the server
> to
> > rule out anything we might be doing wrong.
> >
> >
> > --- stack <st...@duboce.net> wrote:
> >
> >
> >> You are using the shell to do your fetching? Try
> >> writing a little java
> >> program.
> >> St.Ack
> >>
> >>
> >> Charles Kaminski wrote:
> >>
> >>> Hi All,
> >>>
> >>> We're running into sever performance issues.
> I'm
> >>> hoping that there is something simple we can do
> to
> >>> resolve the issues. Any help would be
> >>>
> >> appreciated.
> >>
> >>> Here's what we did:
> >>> 1. Loaded 1,000 records into a table with only
> >>>
> >> two
> >>
> >>> columns - row and content:. Row data is 12
> bytes
> >>>
> >> and
> >>
> >>> content: data is 23 bytes long.
> >>> 2. Using HBase, selected a single record based
> on
> >>>
> >> row
> >>
> >>> in the where clause. Did this for a few
> different
> >>> records. Performance was consistantly 0.01
> >>>
> >> seconds as
> >>
> >>> reported by Hbase.
> >>> 3. Loaded 1,000,000 records into the same table.
>
> >>>
> >> This
> >>
> >>> took 248 seconds using random row values.
> >>> 4. Ran the exact same select statments again as
> in
> >>> step 2. These consistantly took 2 to 3 seconds
> to
> >>> return a single record.
> >>>
> >>> 2 to 3 seconds to return a single record using a
> >>>
> >> key
> >>
> >>> value suggests a major issue with our setup.
> I'm
> >>> hoping you agree and can point us to something
> >>>
> >> we're
> >>
> >>> doing wrong.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >
>
____________________________________________________________________________________
> >
> >>> Looking for last minute shopping deals?
> >>> Find them fast with Yahoo! Search.
> >>>
> >
>
http://tools.search.yahoo.com/newsearch/category.php?category=shopping
> >
> >>>
> >>>
> >>
> >
> >
> >
> >
>
____________________________________________________________________________________
> > Be a better friend, newshound, and
> > know-it-all with Yahoo! Mobile. Try it now.
>
http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
>
> >
> >
>
>
____________________________________________________________________________________
Looking for last minute shopping deals?
Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
Re: Evaluating HBase 3
Posted by stack <st...@duboce.net>.
Lets try and figure out whats going on Charles.
The figures on the end of this page have us random reading bigger values
out of a table of 1M rows at somewhere between 150 and 300 rows a
second, dependent on hbase version (Whats your version?)
Want to send us the code your java apps are using to access hbase so we
can check it out?
Thanks,
St.Ack
Charles Kaminski wrote:
> Hi St.Ack,
>
> Thanks for the response. The performance changes
> below are consistent with what we find in our java
> app. We used Hbase Shell directly on the server to
> rule out anything we might be doing wrong.
>
>
> --- stack <st...@duboce.net> wrote:
>
>
>> You are using the shell to do your fetching? Try
>> writing a little java
>> program.
>> St.Ack
>>
>>
>> Charles Kaminski wrote:
>>
>>> Hi All,
>>>
>>> We're running into sever performance issues. I'm
>>> hoping that there is something simple we can do to
>>> resolve the issues. Any help would be
>>>
>> appreciated.
>>
>>> Here's what we did:
>>> 1. Loaded 1,000 records into a table with only
>>>
>> two
>>
>>> columns - row and content:. Row data is 12 bytes
>>>
>> and
>>
>>> content: data is 23 bytes long.
>>> 2. Using HBase, selected a single record based on
>>>
>> row
>>
>>> in the where clause. Did this for a few different
>>> records. Performance was consistantly 0.01
>>>
>> seconds as
>>
>>> reported by Hbase.
>>> 3. Loaded 1,000,000 records into the same table.
>>>
>> This
>>
>>> took 248 seconds using random row values.
>>> 4. Ran the exact same select statments again as in
>>> step 2. These consistantly took 2 to 3 seconds to
>>> return a single record.
>>>
>>> 2 to 3 seconds to return a single record using a
>>>
>> key
>>
>>> value suggests a major issue with our setup. I'm
>>> hoping you agree and can point us to something
>>>
>> we're
>>
>>> doing wrong.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
> ____________________________________________________________________________________
>
>>> Looking for last minute shopping deals?
>>> Find them fast with Yahoo! Search.
>>>
> http://tools.search.yahoo.com/newsearch/category.php?category=shopping
>
>>>
>>>
>>
>
>
>
> ____________________________________________________________________________________
> Be a better friend, newshound, and
> know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
>
>
Re: Evaluating HBase 3
Posted by Charles Kaminski <fr...@yahoo.com>.
Hi St.Ack,
Thanks for the response. The performance changes
below are consistent with what we find in our java
app. We used Hbase Shell directly on the server to
rule out anything we might be doing wrong.
--- stack <st...@duboce.net> wrote:
> You are using the shell to do your fetching? Try
> writing a little java
> program.
> St.Ack
>
>
> Charles Kaminski wrote:
> > Hi All,
> >
> > We're running into sever performance issues. I'm
> > hoping that there is something simple we can do to
> > resolve the issues. Any help would be
> appreciated.
> >
> > Here's what we did:
> > 1. Loaded 1,000 records into a table with only
> two
> > columns - row and content:. Row data is 12 bytes
> and
> > content: data is 23 bytes long.
> > 2. Using HBase, selected a single record based on
> row
> > in the where clause. Did this for a few different
> > records. Performance was consistantly 0.01
> seconds as
> > reported by Hbase.
> > 3. Loaded 1,000,000 records into the same table.
> This
> > took 248 seconds using random row values.
> > 4. Ran the exact same select statments again as in
> > step 2. These consistantly took 2 to 3 seconds to
> > return a single record.
> >
> > 2 to 3 seconds to return a single record using a
> key
> > value suggests a major issue with our setup. I'm
> > hoping you agree and can point us to something
> we're
> > doing wrong.
> >
> >
> >
> >
> >
> >
> >
>
____________________________________________________________________________________
> > Looking for last minute shopping deals?
> > Find them fast with Yahoo! Search.
>
http://tools.search.yahoo.com/newsearch/category.php?category=shopping
> >
>
>
____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
Re: Evaluating HBase 3
Posted by stack <st...@duboce.net>.
You are using the shell to do your fetching? Try writing a little java
program.
St.Ack
Charles Kaminski wrote:
> Hi All,
>
> We're running into sever performance issues. I'm
> hoping that there is something simple we can do to
> resolve the issues. Any help would be appreciated.
>
> Here's what we did:
> 1. Loaded 1,000 records into a table with only two
> columns - row and content:. Row data is 12 bytes and
> content: data is 23 bytes long.
> 2. Using HBase, selected a single record based on row
> in the where clause. Did this for a few different
> records. Performance was consistantly 0.01 seconds as
> reported by Hbase.
> 3. Loaded 1,000,000 records into the same table. This
> took 248 seconds using random row values.
> 4. Ran the exact same select statments again as in
> step 2. These consistantly took 2 to 3 seconds to
> return a single record.
>
> 2 to 3 seconds to return a single record using a key
> value suggests a major issue with our setup. I'm
> hoping you agree and can point us to something we're
> doing wrong.
>
>
>
>
>
>
> ____________________________________________________________________________________
> Looking for last minute shopping deals?
> Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
>