You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by ch huang <ju...@gmail.com> on 2013/07/01 05:28:37 UTC
different test result from PerformanceEvaluation and from my own test!
i am benchmarking the sequentialWrite speed use HBase,but i find the
result is different ,i do not know which will be right
here is my own procedure,( i modified the PerformanceEvaluation)
void testInsert(String tablename, String[] cfg) {
try{
HTable table = new HTable(conf, tablename);
System.out.println("test begin");
for (int i = 0; i< 1024*1024; i++){
Put put = new Put(Bytes.toBytes("rows"+i));
byte[] value = generateValue(this.rand);
for (int j = 0; j < cfg.length; j++) {
put.add(Bytes.toBytes(cfg[j]),Bytes.toBytes("j"), value);
}
put.setWriteToWAL(writeToWAL);
table.put(put);
if( i > 0 && (i % (1024)) == 0){
System.out.print((i*100/1024/1024) +" percent finished"+"
i="+i+"\n");
}
}
} catch (IOException e){
e.printStackTrace();
}
}
public static void main (String [] agrs) {
try {
String tablename = "demo2";
String[] familys = { "grade", "course" };
// HBaseTest. creatTable(tablename, familys);
HBaseTest ht = new HBaseTest();
long startTime = System.currentTimeMillis();
ht.testInsert(tablename,familys);
long elapsedTime = System.currentTimeMillis() - startTime;
System.out.println("elapsed time is : "+elapsedTime+" ms\n");
} catch (Exception e) {
e.printStackTrace();
}
}
and result is
elapsed time is : 1956328 ms
also i use PerformanceEvaluation ,BTW: i am newbie on hbase,so the whole
system is default ,not tune yet
# hbase org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite 1
13/07/01 09:08:34 INFO hbase.PerformanceEvaluation: Finished class
org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest in
81466ms at offset 0 for 1048576 rows
Re: different test result from PerformanceEvaluation and from my own test!
Posted by Ted Yu <yu...@gmail.com>.
bq. String[] familys = { "grade", "course" };
...
bq. for (int j = 0; j < cfg.length; j++) {
You were generating more data in the Put's for testInsert().
Cheers
On Mon, Jul 1, 2013 at 11:28 AM, ch huang <ju...@gmail.com> wrote:
> i am benchmarking the sequentialWrite speed use HBase,but i find the
> result is different ,i do not know which will be right
>
> here is my own procedure,( i modified the PerformanceEvaluation)
>
> void testInsert(String tablename, String[] cfg) {
> try{
> HTable table = new HTable(conf, tablename);
> System.out.println("test begin");
> for (int i = 0; i< 1024*1024; i++){
> Put put = new Put(Bytes.toBytes("rows"+i));
> byte[] value = generateValue(this.rand);
> for (int j = 0; j < cfg.length; j++) {
> put.add(Bytes.toBytes(cfg[j]),Bytes.toBytes("j"), value);
> }
> put.setWriteToWAL(writeToWAL);
> table.put(put);
> if( i > 0 && (i % (1024)) == 0){
> System.out.print((i*100/1024/1024) +" percent finished"+"
> i="+i+"\n");
> }
> }
>
> } catch (IOException e){
> e.printStackTrace();
> }
>
> }
>
> public static void main (String [] agrs) {
> try {
> String tablename = "demo2";
> String[] familys = { "grade", "course" };
> // HBaseTest. creatTable(tablename, familys);
> HBaseTest ht = new HBaseTest();
> long startTime = System.currentTimeMillis();
> ht.testInsert(tablename,familys);
>
> long elapsedTime = System.currentTimeMillis() - startTime;
>
> System.out.println("elapsed time is : "+elapsedTime+" ms\n");
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
>
>
> and result is
> elapsed time is : 1956328 ms
>
> also i use PerformanceEvaluation ,BTW: i am newbie on hbase,so the whole
> system is default ,not tune yet
>
> # hbase org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite 1
>
> 13/07/01 09:08:34 INFO hbase.PerformanceEvaluation: Finished class
> org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest in
> 81466ms at offset 0 for 1048576 rows
>