You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2012/05/02 09:53:01 UTC
[jira] [Created] (HBASE-5915) Improve performance of KeyValue
comparison
Todd Lipcon created HBASE-5915:
----------------------------------
Summary: Improve performance of KeyValue comparison
Key: HBASE-5915
URL: https://issues.apache.org/jira/browse/HBASE-5915
Project: HBase
Issue Type: Improvement
Components: performance
Affects Versions: 0.94.1
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266662#comment-13266662 ]
stack commented on HBASE-5915:
------------------------------
It looks great Todd.
This is funny (when wrong-endian) --> + return Long.reverseBytes(result);
Nice --> + final int minWords = minLength >>> 3; // / SIZEOF_LONG;.... that should be faster.
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266708#comment-13266708 ]
ramkrishna.s.vasudevan commented on HBASE-5915:
-----------------------------------------------
{code}
+ return lessThanUnsigned(Long.reverseBytes(lw),
+ Long.reverseBytes(rw)) ? -1 : 1;
{code}
Patch v2 is different from V1. Was it intended Ted?
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: 5915-v2.txt, hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "Zhihong Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266711#comment-13266711 ]
Zhihong Yu commented on HBASE-5915:
-----------------------------------
TestKVComparisonPerformance without the patch:
{code}
Run 99 took 99574us
Avg: 98655us
{code}
Here is the output with the patch:
{code}
Run 99 took 79849us
Avg: 79295us
{code}
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: 5915-v2.txt, hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "Zhihong Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266575#comment-13266575 ]
Zhihong Yu commented on HBASE-5915:
-----------------------------------
The following variable doesn't seem to be used:
{code}
+ int triggerBoundsCheck = buf[off + SIZEOF_LONG - 1];
{code}
What's its purpose ?
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266823#comment-13266823 ]
Hadoop QA commented on HBASE-5915:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12525313/5915-v2.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 hadoop23. The patch compiles against the hadoop 0.23.x profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1728//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1728//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1728//console
This message is automatically generated.
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: 5915-v2.txt, hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266755#comment-13266755 ]
Todd Lipcon commented on HBASE-5915:
------------------------------------
Before we commit this, we should run some actual workloads. There are two funky things I noticed about the patch:
1) the test case always has the behavior that the first KV to be compared is in cache while the second one is not, which is not quite realistic in a real workload. Maybe it's better to time insertion of a few million KVs into a KeyValueSkipListSet instead.
2) when I tried this patch with a YCSB workload, it actually took _more_ CPU seconds than without the patch, which is odd given the microbenchmark. So something isn't aligning with reality. I didn't have time to investigate that.
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: 5915-v2.txt, hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "Zhihong Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266716#comment-13266716 ]
Zhihong Yu commented on HBASE-5915:
-----------------------------------
@Ram:
The quoted snippet came from Todd.
Can you elaborate what problem you saw ?
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: 5915-v2.txt, hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266682#comment-13266682 ]
Hadoop QA commented on HBASE-5915:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12525264/hbase-5915.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified tests.
+1 hadoop23. The patch compiles against the hadoop 0.23.x profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.TestCheckTestClasses
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1726//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1726//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1726//console
This message is automatically generated.
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266720#comment-13266720 ]
ramkrishna.s.vasudevan commented on HBASE-5915:
-----------------------------------------------
@Ted
Sorry Ted. I think its fine. Once i applied the patch it looked fine. Sorry for the confusion.
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: 5915-v2.txt, hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "Zhihong Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5915:
------------------------------
Attachment: 5915-v2.txt
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: 5915-v2.txt, hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "Zhihong Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5915:
------------------------------
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5915) Improve performance of KeyValue
comparison
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HBASE-5915:
-------------------------------
Attachment: hbase-5915.txt
Here's the patch and micro-benchmark.
Would be curious if people can poke holes in it (or show with a more realistic workload if it makes a difference)
> Improve performance of KeyValue comparison
> ------------------------------------------
>
> Key: HBASE-5915
> URL: https://issues.apache.org/jira/browse/HBASE-5915
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.94.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hbase-5915.txt
>
>
> Got bored tonight and poked around at making KeyValue comparison faster. I was able to get a ~23% speedup on a micro-benchmark I wrote by using Unsafe in a few more places. This should make some difference for real workloads, since this is usually one of the top CPU consumers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira