You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "ryan rawson (JIRA)" <ji...@apache.org> on 2010/10/02 01:55:32 UTC
[jira] Created: (HBASE-3073) New APIs for Result, faster
implementation for some calls
New APIs for Result, faster implementation for some calls
---------------------------------------------------------
Key: HBASE-3073
URL: https://issues.apache.org/jira/browse/HBASE-3073
Project: HBase
Issue Type: Bug
Affects Versions: 0.89.20100924
Reporter: ryan rawson
Assignee: ryan rawson
Fix For: 0.90.0
Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
I propose the following new APIs:
public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
public boolean containsColumn(byte [] family, byte [] qualifier);
public byte [] getValue(byte [] family, byte [] qualifier);
Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917128#action_12917128 ]
ryan rawson commented on HBASE-3073:
------------------------------------
this does indeed fix HBASE-1937, and HBASE-2753 is in progress under it's own commit. At that time we can make raw/sorted do the same thing (just return 'kvs').
This fixes HBASE-1937 by introducing the getColumnLatest() which returns a KeyValue which the user can call getTimestamp() on. Instead of creating Result API calls for every conceivable thing a user might want to do, let's just expose the KeyValue which has a rich API for doing all sorts of things, such as getting various fields (timestamp, qualifier, field, row, value) and comparison.
I need to do a bit more perf testing and then I will clean up those javadocs.
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918284#action_12918284 ]
ryan rawson commented on HBASE-3073:
------------------------------------
i beefed up the javadoc a lot. I removed the 'ex' and just replaced the older implementations since my testing shows that the new code is faster or as fast as the old in all situations.
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson updated HBASE-3073:
-------------------------------
Attachment: HBASE-3073.txt
updated patch removing some WIP stuff that doesnt make sense
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917125#action_12917125 ]
stack commented on HBASE-3073:
------------------------------
On this patch, if it fixed HBASE-1937 and HBASE-2753 it'd help your case that the fix for this issue is 'vital'.
Whats this?
{code}
+ //private transient int size = 0; // size of underlying data (kv.getLength all added together)
{code}
Remove it?
For this change....
{code}
/**
- * Return the unsorted array of KeyValues backing this Result instance.
- * @return unsorted array of KeyValues
+ * Return the array of KeyValues backing this Result instance.
+ * @return array of KeyValues
*/
{code]
... is the result actually sorted? If so, javadoc should say so?
Why we sorting (see HBASE-2753)?
{code}
public KeyValue[] sorted() {
- if (isEmpty()) { // used for side effect!
- return null;
- }
+ raw(); // side effect of loading this.kvs
if (!sorted) {
{code}
...
Below .....
{code}
+ * @param family
+ * @param qualifier
+ * @return
+ */
{code}
fix the javadoc... fix the return at least. It looks like if no such column you don't get null but an empty List. Should note that.
Need to fix javadoc in these highly visible methods elsewhere in the patch too.
Do these new APIs replace others? Are there equivs in the API already for these? If so, deprecate the old and link in javadoc else new APIs to do same thing confuses.
Fix these names:
valueEx
and
containsColumnEx
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Updated: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by Ryan Rawson <ry...@gmail.com>.
Le fixed!
-ryan
On Tue, Oct 5, 2010 at 7:00 PM, Andrew Purtell <ap...@apache.org> wrote:
> This seems to have broken a number of REST tests. IndexOutOfBoundsException from Result#getValue out of o.a.h.h.rest.TestScannersWithFilters, etc.
>
> Best regards,
>
> - Andy
>
>
> --- On Tue, 10/5/10, ryan rawson (JIRA) <ji...@apache.org> wrote:
>
>> From: ryan rawson (JIRA) <ji...@apache.org>
>> Subject: [jira] Updated: (HBASE-3073) New APIs for Result, faster implementation for some calls
>> To: issues@hbase.apache.org
>> Date: Tuesday, October 5, 2010, 5:45 PM
>>
>> [ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>> ]
>>
>> ryan rawson updated HBASE-3073:
>> -------------------------------
>>
>> Resolution: Fixed
>> Status: Resolved (was:
>> Patch Available)
>>
>> i committed this. thanks to all the reviewers!
>
>
>
>
>
>
Re: [jira] Updated: (HBASE-3073) New APIs for Result, faster implementation for some calls
Posted by Andrew Purtell <ap...@apache.org>.
-------------------------------------------------------------------------------
Test set: org.apache.hadoop.hbase.rest.TestScannersWithFilters
-------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 20.781 sec <<< FAILURE!
testScannersWithFilters(org.apache.hadoop.hbase.rest.TestScannersWithFilters) Time elapsed: 20.759 sec <<< ERROR!
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.hadoop.hbase.client.Result.getColumnLatest(Result.java:265)
at org.apache.hadoop.hbase.client.Result.getValue(Result.java:279)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:690)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:536)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:507)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:166)
at org.apache.hadoop.hbase.HBaseClusterTestCase.hBaseClusterSetup(HBaseClusterTestCase.java:111)
at org.apache.hadoop.hbase.HBaseClusterTestCase.setUp(HBaseClusterTestCase.java:148)
at org.apache.hadoop.hbase.rest.HBaseRESTClusterTestBase.setUp(HBaseRESTClusterTestBase.java:43)
at org.apache.hadoop.hbase.rest.TestScannersWithFilters.setUp(TestScannersWithFilters.java:110)
at junit.framework.TestCase.runBare(TestCase.java:132)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:62)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:165)
at org.apache.maven.surefire.Surefire.run(Surefire.java:107)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:289)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1005)
--- On Tue, 10/5/10, Andrew Purtell <ap...@apache.org> wrote:
> From: Andrew Purtell <ap...@apache.org>
> Subject: Re: [jira] Updated: (HBASE-3073) New APIs for Result, faster implementation for some calls
> To: dev@hbase.apache.org
> Date: Tuesday, October 5, 2010, 7:00 PM
> This seems to have broken a number of
> REST tests. IndexOutOfBoundsException from Result#getValue
> out of o.a.h.h.rest.TestScannersWithFilters, etc.
>
> Best regards,
>
> - Andy
>
>
> --- On Tue, 10/5/10, ryan rawson (JIRA) <ji...@apache.org>
> wrote:
>
> > From: ryan rawson (JIRA) <ji...@apache.org>
> > Subject: [jira] Updated: (HBASE-3073) New APIs for
> Result, faster implementation for some calls
> > To: issues@hbase.apache.org
> > Date: Tuesday, October 5, 2010, 5:45 PM
> >
> > [ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > ]
> >
> > ryan rawson updated HBASE-3073:
> > -------------------------------
> >
> > Resolution: Fixed
> > Status: Resolved (was:
> > Patch Available)
> >
> > i committed this. thanks to all the reviewers!
>
>
>
>
>
>
Re: [jira] Updated: (HBASE-3073) New APIs for Result, faster implementation for some calls
Posted by Andrew Purtell <ap...@apache.org>.
This seems to have broken a number of REST tests. IndexOutOfBoundsException from Result#getValue out of o.a.h.h.rest.TestScannersWithFilters, etc.
Best regards,
- Andy
--- On Tue, 10/5/10, ryan rawson (JIRA) <ji...@apache.org> wrote:
> From: ryan rawson (JIRA) <ji...@apache.org>
> Subject: [jira] Updated: (HBASE-3073) New APIs for Result, faster implementation for some calls
> To: issues@hbase.apache.org
> Date: Tuesday, October 5, 2010, 5:45 PM
>
> [ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
>
> ryan rawson updated HBASE-3073:
> -------------------------------
>
> Resolution: Fixed
> Status: Resolved (was:
> Patch Available)
>
> i committed this. thanks to all the reviewers!
[jira] Updated: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson updated HBASE-3073:
-------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
i committed this. thanks to all the reviewers!
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918339#action_12918339 ]
HBase Review Board commented on HBASE-3073:
-------------------------------------------
Message from: "Jonathan Gray" <jg...@apache.org>
bq. On 2010-10-05 16:25:58, Ryan Rawson wrote:
bq. > trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java, line 202
bq. > <http://review.cloudera.org/r/963/diff/1/?file=14002#file14002line202>
bq. >
bq. > after https://issues.apache.org/jira/browse/HBASE-2753 is resolved, raw()===sort() and there wont be a difference.
Okay. Just saying, weird to use a deprecated method internally but agree that it doesn't matter if you're planning to remove the assert.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/963/#review1428
-----------------------------------------------------------
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918365#action_12918365 ]
HBase Review Board commented on HBASE-3073:
-------------------------------------------
Message from: "Ryan Rawson" <ry...@gmail.com>
bq. On 2010-10-05 16:25:58, Ryan Rawson wrote:
bq. > trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java, line 202
bq. > <http://review.cloudera.org/r/963/diff/1/?file=14002#file14002line202>
bq. >
bq. > after https://issues.apache.org/jira/browse/HBASE-2753 is resolved, raw()===sort() and there wont be a difference.
bq.
bq. Jonathan Gray wrote:
bq. Okay. Just saying, weird to use a deprecated method internally but agree that it doesn't matter if you're planning to remove the assert.
the code calls raw() now, hopefully our asserts dont trigger
- Ryan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/963/#review1428
-----------------------------------------------------------
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson updated HBASE-3073:
-------------------------------
Attachment: HBASE-3073.txt
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917115#action_12917115 ]
Andrew Purtell commented on HBASE-3073:
---------------------------------------
+1 based on the YCSB improvement.
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson updated HBASE-3073:
-------------------------------
Status: Patch Available (was: Open)
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918336#action_12918336 ]
HBase Review Board commented on HBASE-3073:
-------------------------------------------
Message from: "Ryan Rawson" <ry...@gmail.com>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/963/#review1428
-----------------------------------------------------------
trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java
<http://review.cloudera.org/r/963/#comment4989>
after https://issues.apache.org/jira/browse/HBASE-2753 is resolved, raw()===sort() and there wont be a difference.
- Ryan
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson updated HBASE-3073:
-------------------------------
Attachment: (was: HBASE-3073.txt)
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918303#action_12918303 ]
HBase Review Board commented on HBASE-3073:
-------------------------------------------
Message from: "Ryan Rawson" <ry...@gmail.com>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/963/#review1426
-----------------------------------------------------------
trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java
<http://review.cloudera.org/r/963/#comment4980>
i removed this in my svn client, it was remnant of my perf testing
- Ryan
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918321#action_12918321 ]
HBase Review Board commented on HBASE-3073:
-------------------------------------------
Message from: "Jonathan Gray" <jg...@apache.org>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/963/#review1427
-----------------------------------------------------------
Just javadoc cleanup, otherwise this looks great. As noted in comments, I think we should deprecate sorted(), recommend and @link to raw(), and ensure we aren't using sorted() internally (especially since it actually is less efficient w/ the assert).
trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java
<http://review.cloudera.org/r/963/#comment4982>
You might {@link to KV.Comparator so javadoc for that can show up on hover
trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java
<http://review.cloudera.org/r/963/#comment4981>
> 80
trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java
<http://review.cloudera.org/r/963/#comment4983>
Add @deprecated and then recommend using raw instead w/ an @link to it
trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java
<http://review.cloudera.org/r/963/#comment4984>
Should be "Returns the" or "Return the" rather than 'me'. Also >80 chars
trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java
<http://review.cloudera.org/r/963/#comment4985>
@link
trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java
<http://review.cloudera.org/r/963/#comment4986>
Should use raw()?
trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java
<http://review.cloudera.org/r/963/#comment4987>
>80
trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java
<http://review.cloudera.org/r/963/#comment4988>
Maybe "Returns (or Gets) the KeyValue of the most recent version of the specified column"... and >80
- Jonathan
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3073) New APIs for Result, faster
implementation for some calls
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918297#action_12918297 ]
HBase Review Board commented on HBASE-3073:
-------------------------------------------
Message from: "Ryan Rawson" <ry...@gmail.com>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/963/
-----------------------------------------------------------
Review request for hbase.
Summary
-------
New APIs for Result, faster implementation for some calls
This addresses bug HBASE-3073.
http://issues.apache.org/jira/browse/HBASE-3073
Diffs
-----
trunk/CHANGES.txt 1003710
trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java 1003710
trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java 1003709
trunk/src/test/java/org/apache/hadoop/hbase/client/TestResult.java PRE-CREATION
Diff: http://review.cloudera.org/r/963/diff
Testing
-------
Thanks,
Ryan
> New APIs for Result, faster implementation for some calls
> ---------------------------------------------------------
>
> Key: HBASE-3073
> URL: https://issues.apache.org/jira/browse/HBASE-3073
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100924
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
> Attachments: HBASE-3073.txt
>
>
> Our existing API for Result hasn't been given much love in the last year. In the mean time, inefficiencies in the existing implementation have come to light, causing issues with benchmarks. Furthermore, some people are finding the API both difficult to use as well as not useful enough (See: HBASE-1937).
> I propose the following new APIs:
> public List<KeyValue> getColumn(byte [] family, byte [] qualifier);
> public KeyValue getColumnLatest(byte [] family, byte [] qualifier);
> The implementation of these use a binary search on the underlying kvs array (which is sorted). I also have new implementations for
> public boolean containsColumn(byte [] family, byte [] qualifier);
> public byte [] getValue(byte [] family, byte [] qualifier);
> Which in the small case run faster, but in the big case seem to run a bit slower. That is if you call getValue() 10 times for a Result it will be faster with the new implementation, but if you call getValue() 100 times for the same Result it is faster using the old implementation. My tests indicated about 10% slower on 'getValue' 100x with an overall 1000x iteration on 1000 different Result objects. Considering most people use getValue() to retrieve named columns and iteration when the qualifier list is unknown I think this is a reasonable trade off.
> Along with the new API, there is a recommendation to use raw() to get the list of KeyValue objects for iteration. This increases the visibility of KeyValue, and also is much faster to iterate (4.9 times on my mini benchmark, 100 columns per Result, redone 1000 times on different Result objects).
> Given my recent major speed boost by changing YCSB to use the raw() interface, I think that this is a must have for 0.90.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.