You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Thanh Do <th...@cs.wisc.edu> on 2013/06/02 17:51:57 UTC
read test with PE
hi,
I tried to run read test with PerformanceEvaluation (PE).
What I did was.
1) create TestTable with a bunch of key-value first using PE sequentialWrite
2) run PE randomRead using following command.
hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
However, from the log, it seems that at step 2, PE actually disables,
deletes
and recreates the TestTable before running the randomRead test.
This means that the read test runs on an empty table!
Is this an expected behavior, or did I do something wrong?
Thanks,
Thanh
Re: read test with PE
Posted by Thanh Do <th...@cs.wisc.edu>.
Thanks Ted for looking into this.
I think the presplit opiton is the problem.
In step 1, I created the table with --presplit=3
In step 2, I run the read test and passed --presplit option as well.
As a result, the deletion happened during step 2.
Removing --presplit option at step 2 solved the problem.
Thanh
On Sun, Jun 2, 2013 at 11:02 AM, Ted Yu <yu...@gmail.com> wrote:
> Looking at the tip of 0.94:
>
> private boolean checkTable(HBaseAdmin admin) throws IOException {
> HTableDescriptor tableDescriptor = getTableDescriptor();
> if (this.presplitRegions > 0) {
> // presplit requested
> if (admin.tableExists(tableDescriptor.getName())) {
> admin.disableTable(tableDescriptor.getName());
> admin.deleteTable(tableDescriptor.getName());
> }
> This means that if you specified --presplit=XX option in command line,
> pre-existing table would be dropped.
>
> Looking at the rev history
> for src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java, there
> was no change since 2012-02-27 (branch of 0.94)
>
> Can you check whether table deletion really happened during step 2 ?
>
> Cheers
>
> On Sun, Jun 2, 2013 at 8:55 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>
>> and my hbase version is 0.94.2
>>
>>
>> On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>>
>>> hi,
>>>
>>> I tried to run read test with PerformanceEvaluation (PE).
>>> What I did was.
>>> 1) create TestTable with a bunch of key-value first using PE
>>> sequentialWrite
>>> 2) run PE randomRead using following command.
>>> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>>>
>>> However, from the log, it seems that at step 2, PE actually disables,
>>> deletes
>>> and recreates the TestTable before running the randomRead test.
>>> This means that the read test runs on an empty table!
>>>
>>> Is this an expected behavior, or did I do something wrong?
>>>
>>> Thanks,
>>> Thanh
>>>
>>>
>>>
>>>
>>
>
Re: read test with PE
Posted by Thanh Do <th...@cs.wisc.edu>.
Thanks Ted for looking into this.
I think the presplit opiton is the problem.
In step 1, I created the table with --presplit=3
In step 2, I run the read test and passed --presplit option as well.
As a result, the deletion happened during step 2.
Removing --presplit option at step 2 solved the problem.
Thanh
On Sun, Jun 2, 2013 at 11:02 AM, Ted Yu <yu...@gmail.com> wrote:
> Looking at the tip of 0.94:
>
> private boolean checkTable(HBaseAdmin admin) throws IOException {
> HTableDescriptor tableDescriptor = getTableDescriptor();
> if (this.presplitRegions > 0) {
> // presplit requested
> if (admin.tableExists(tableDescriptor.getName())) {
> admin.disableTable(tableDescriptor.getName());
> admin.deleteTable(tableDescriptor.getName());
> }
> This means that if you specified --presplit=XX option in command line,
> pre-existing table would be dropped.
>
> Looking at the rev history
> for src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java, there
> was no change since 2012-02-27 (branch of 0.94)
>
> Can you check whether table deletion really happened during step 2 ?
>
> Cheers
>
> On Sun, Jun 2, 2013 at 8:55 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>
>> and my hbase version is 0.94.2
>>
>>
>> On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>>
>>> hi,
>>>
>>> I tried to run read test with PerformanceEvaluation (PE).
>>> What I did was.
>>> 1) create TestTable with a bunch of key-value first using PE
>>> sequentialWrite
>>> 2) run PE randomRead using following command.
>>> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>>>
>>> However, from the log, it seems that at step 2, PE actually disables,
>>> deletes
>>> and recreates the TestTable before running the randomRead test.
>>> This means that the read test runs on an empty table!
>>>
>>> Is this an expected behavior, or did I do something wrong?
>>>
>>> Thanks,
>>> Thanh
>>>
>>>
>>>
>>>
>>
>
Re: read test with PE
Posted by Thanh Do <th...@cs.wisc.edu>.
Thanks Ted for looking into this.
I think the presplit opiton is the problem.
In step 1, I created the table with --presplit=3
In step 2, I run the read test and passed --presplit option as well.
As a result, the deletion happened during step 2.
Removing --presplit option at step 2 solved the problem.
Thanh
On Sun, Jun 2, 2013 at 11:02 AM, Ted Yu <yu...@gmail.com> wrote:
> Looking at the tip of 0.94:
>
> private boolean checkTable(HBaseAdmin admin) throws IOException {
> HTableDescriptor tableDescriptor = getTableDescriptor();
> if (this.presplitRegions > 0) {
> // presplit requested
> if (admin.tableExists(tableDescriptor.getName())) {
> admin.disableTable(tableDescriptor.getName());
> admin.deleteTable(tableDescriptor.getName());
> }
> This means that if you specified --presplit=XX option in command line,
> pre-existing table would be dropped.
>
> Looking at the rev history
> for src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java, there
> was no change since 2012-02-27 (branch of 0.94)
>
> Can you check whether table deletion really happened during step 2 ?
>
> Cheers
>
> On Sun, Jun 2, 2013 at 8:55 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>
>> and my hbase version is 0.94.2
>>
>>
>> On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>>
>>> hi,
>>>
>>> I tried to run read test with PerformanceEvaluation (PE).
>>> What I did was.
>>> 1) create TestTable with a bunch of key-value first using PE
>>> sequentialWrite
>>> 2) run PE randomRead using following command.
>>> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>>>
>>> However, from the log, it seems that at step 2, PE actually disables,
>>> deletes
>>> and recreates the TestTable before running the randomRead test.
>>> This means that the read test runs on an empty table!
>>>
>>> Is this an expected behavior, or did I do something wrong?
>>>
>>> Thanks,
>>> Thanh
>>>
>>>
>>>
>>>
>>
>
Re: read test with PE
Posted by Thanh Do <th...@cs.wisc.edu>.
Thanks Ted for looking into this.
I think the presplit opiton is the problem.
In step 1, I created the table with --presplit=3
In step 2, I run the read test and passed --presplit option as well.
As a result, the deletion happened during step 2.
Removing --presplit option at step 2 solved the problem.
Thanh
On Sun, Jun 2, 2013 at 11:02 AM, Ted Yu <yu...@gmail.com> wrote:
> Looking at the tip of 0.94:
>
> private boolean checkTable(HBaseAdmin admin) throws IOException {
> HTableDescriptor tableDescriptor = getTableDescriptor();
> if (this.presplitRegions > 0) {
> // presplit requested
> if (admin.tableExists(tableDescriptor.getName())) {
> admin.disableTable(tableDescriptor.getName());
> admin.deleteTable(tableDescriptor.getName());
> }
> This means that if you specified --presplit=XX option in command line,
> pre-existing table would be dropped.
>
> Looking at the rev history
> for src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java, there
> was no change since 2012-02-27 (branch of 0.94)
>
> Can you check whether table deletion really happened during step 2 ?
>
> Cheers
>
> On Sun, Jun 2, 2013 at 8:55 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>
>> and my hbase version is 0.94.2
>>
>>
>> On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>>
>>> hi,
>>>
>>> I tried to run read test with PerformanceEvaluation (PE).
>>> What I did was.
>>> 1) create TestTable with a bunch of key-value first using PE
>>> sequentialWrite
>>> 2) run PE randomRead using following command.
>>> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>>>
>>> However, from the log, it seems that at step 2, PE actually disables,
>>> deletes
>>> and recreates the TestTable before running the randomRead test.
>>> This means that the read test runs on an empty table!
>>>
>>> Is this an expected behavior, or did I do something wrong?
>>>
>>> Thanks,
>>> Thanh
>>>
>>>
>>>
>>>
>>
>
Re: read test with PE
Posted by Ted Yu <yu...@gmail.com>.
Looking at the tip of 0.94:
private boolean checkTable(HBaseAdmin admin) throws IOException {
HTableDescriptor tableDescriptor = getTableDescriptor();
if (this.presplitRegions > 0) {
// presplit requested
if (admin.tableExists(tableDescriptor.getName())) {
admin.disableTable(tableDescriptor.getName());
admin.deleteTable(tableDescriptor.getName());
}
This means that if you specified --presplit=XX option in command line,
pre-existing table would be dropped.
Looking at the rev history
for src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java, there
was no change since 2012-02-27 (branch of 0.94)
Can you check whether table deletion really happened during step 2 ?
Cheers
On Sun, Jun 2, 2013 at 8:55 AM, Thanh Do <th...@cs.wisc.edu> wrote:
> and my hbase version is 0.94.2
>
>
> On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>
>> hi,
>>
>> I tried to run read test with PerformanceEvaluation (PE).
>> What I did was.
>> 1) create TestTable with a bunch of key-value first using PE
>> sequentialWrite
>> 2) run PE randomRead using following command.
>> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>>
>> However, from the log, it seems that at step 2, PE actually disables,
>> deletes
>> and recreates the TestTable before running the randomRead test.
>> This means that the read test runs on an empty table!
>>
>> Is this an expected behavior, or did I do something wrong?
>>
>> Thanks,
>> Thanh
>>
>>
>>
>>
>
Re: read test with PE
Posted by Ted Yu <yu...@gmail.com>.
Looking at the tip of 0.94:
private boolean checkTable(HBaseAdmin admin) throws IOException {
HTableDescriptor tableDescriptor = getTableDescriptor();
if (this.presplitRegions > 0) {
// presplit requested
if (admin.tableExists(tableDescriptor.getName())) {
admin.disableTable(tableDescriptor.getName());
admin.deleteTable(tableDescriptor.getName());
}
This means that if you specified --presplit=XX option in command line,
pre-existing table would be dropped.
Looking at the rev history
for src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java, there
was no change since 2012-02-27 (branch of 0.94)
Can you check whether table deletion really happened during step 2 ?
Cheers
On Sun, Jun 2, 2013 at 8:55 AM, Thanh Do <th...@cs.wisc.edu> wrote:
> and my hbase version is 0.94.2
>
>
> On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>
>> hi,
>>
>> I tried to run read test with PerformanceEvaluation (PE).
>> What I did was.
>> 1) create TestTable with a bunch of key-value first using PE
>> sequentialWrite
>> 2) run PE randomRead using following command.
>> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>>
>> However, from the log, it seems that at step 2, PE actually disables,
>> deletes
>> and recreates the TestTable before running the randomRead test.
>> This means that the read test runs on an empty table!
>>
>> Is this an expected behavior, or did I do something wrong?
>>
>> Thanks,
>> Thanh
>>
>>
>>
>>
>
Re: read test with PE
Posted by Ted Yu <yu...@gmail.com>.
Looking at the tip of 0.94:
private boolean checkTable(HBaseAdmin admin) throws IOException {
HTableDescriptor tableDescriptor = getTableDescriptor();
if (this.presplitRegions > 0) {
// presplit requested
if (admin.tableExists(tableDescriptor.getName())) {
admin.disableTable(tableDescriptor.getName());
admin.deleteTable(tableDescriptor.getName());
}
This means that if you specified --presplit=XX option in command line,
pre-existing table would be dropped.
Looking at the rev history
for src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java, there
was no change since 2012-02-27 (branch of 0.94)
Can you check whether table deletion really happened during step 2 ?
Cheers
On Sun, Jun 2, 2013 at 8:55 AM, Thanh Do <th...@cs.wisc.edu> wrote:
> and my hbase version is 0.94.2
>
>
> On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>
>> hi,
>>
>> I tried to run read test with PerformanceEvaluation (PE).
>> What I did was.
>> 1) create TestTable with a bunch of key-value first using PE
>> sequentialWrite
>> 2) run PE randomRead using following command.
>> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>>
>> However, from the log, it seems that at step 2, PE actually disables,
>> deletes
>> and recreates the TestTable before running the randomRead test.
>> This means that the read test runs on an empty table!
>>
>> Is this an expected behavior, or did I do something wrong?
>>
>> Thanks,
>> Thanh
>>
>>
>>
>>
>
Re: read test with PE
Posted by Ted Yu <yu...@gmail.com>.
Looking at the tip of 0.94:
private boolean checkTable(HBaseAdmin admin) throws IOException {
HTableDescriptor tableDescriptor = getTableDescriptor();
if (this.presplitRegions > 0) {
// presplit requested
if (admin.tableExists(tableDescriptor.getName())) {
admin.disableTable(tableDescriptor.getName());
admin.deleteTable(tableDescriptor.getName());
}
This means that if you specified --presplit=XX option in command line,
pre-existing table would be dropped.
Looking at the rev history
for src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java, there
was no change since 2012-02-27 (branch of 0.94)
Can you check whether table deletion really happened during step 2 ?
Cheers
On Sun, Jun 2, 2013 at 8:55 AM, Thanh Do <th...@cs.wisc.edu> wrote:
> and my hbase version is 0.94.2
>
>
> On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
>
>> hi,
>>
>> I tried to run read test with PerformanceEvaluation (PE).
>> What I did was.
>> 1) create TestTable with a bunch of key-value first using PE
>> sequentialWrite
>> 2) run PE randomRead using following command.
>> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>>
>> However, from the log, it seems that at step 2, PE actually disables,
>> deletes
>> and recreates the TestTable before running the randomRead test.
>> This means that the read test runs on an empty table!
>>
>> Is this an expected behavior, or did I do something wrong?
>>
>> Thanks,
>> Thanh
>>
>>
>>
>>
>
Re: read test with PE
Posted by Thanh Do <th...@cs.wisc.edu>.
and my hbase version is 0.94.2
On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
> hi,
>
> I tried to run read test with PerformanceEvaluation (PE).
> What I did was.
> 1) create TestTable with a bunch of key-value first using PE
> sequentialWrite
> 2) run PE randomRead using following command.
> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>
> However, from the log, it seems that at step 2, PE actually disables,
> deletes
> and recreates the TestTable before running the randomRead test.
> This means that the read test runs on an empty table!
>
> Is this an expected behavior, or did I do something wrong?
>
> Thanks,
> Thanh
>
>
>
>
Re: read test with PE
Posted by Thanh Do <th...@cs.wisc.edu>.
and my hbase version is 0.94.2
On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
> hi,
>
> I tried to run read test with PerformanceEvaluation (PE).
> What I did was.
> 1) create TestTable with a bunch of key-value first using PE
> sequentialWrite
> 2) run PE randomRead using following command.
> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>
> However, from the log, it seems that at step 2, PE actually disables,
> deletes
> and recreates the TestTable before running the randomRead test.
> This means that the read test runs on an empty table!
>
> Is this an expected behavior, or did I do something wrong?
>
> Thanks,
> Thanh
>
>
>
>
Re: read test with PE
Posted by Thanh Do <th...@cs.wisc.edu>.
and my hbase version is 0.94.2
On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
> hi,
>
> I tried to run read test with PerformanceEvaluation (PE).
> What I did was.
> 1) create TestTable with a bunch of key-value first using PE
> sequentialWrite
> 2) run PE randomRead using following command.
> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>
> However, from the log, it seems that at step 2, PE actually disables,
> deletes
> and recreates the TestTable before running the randomRead test.
> This means that the read test runs on an empty table!
>
> Is this an expected behavior, or did I do something wrong?
>
> Thanks,
> Thanh
>
>
>
>
Re: read test with PE
Posted by Thanh Do <th...@cs.wisc.edu>.
and my hbase version is 0.94.2
On Sun, Jun 2, 2013 at 10:51 AM, Thanh Do <th...@cs.wisc.edu> wrote:
> hi,
>
> I tried to run read test with PerformanceEvaluation (PE).
> What I did was.
> 1) create TestTable with a bunch of key-value first using PE
> sequentialWrite
> 2) run PE randomRead using following command.
> hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead 3
>
> However, from the log, it seems that at step 2, PE actually disables,
> deletes
> and recreates the TestTable before running the randomRead test.
> This means that the read test runs on an empty table!
>
> Is this an expected behavior, or did I do something wrong?
>
> Thanks,
> Thanh
>
>
>
>