You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Smidth Panchamia <sp...@maprtech.com> on 2015/07/16 23:07:30 UTC
Review Request 36558: DRILL-3492: Add support for encoding/decoding of
'double' to/from OrderedBytes format
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36558/
-----------------------------------------------------------
Review request for drill and Aditya Kishore.
Bugs: DRILL-3492
https://issues.apache.org/jira/browse/DRILL-3492
Repository: drill-git
Description
-------
This change allows encoding/decoding of data from Double data type to OrderedBytes format.
The following JIRA added the OrderedBytes encoding to HBase: https://issues.apache.org/jira/browse/HBASE-8201
This encoding scheme will preserve the sort-order of the native data-type when it is stored as sorted byte arrays on disk.
Thus, it will help the HBase storage plugin if the row-keys have been encoded in OrderedBytes format.
This functionality allows us to prune the scan ranges, thus reading much lesser data from the server.
Diffs
-----
contrib/storage-hbase/pom.xml d027771
contrib/storage-hbase/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/OrderedBytesDoubleConvertFrom.java PRE-CREATION
contrib/storage-hbase/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/OrderedBytesDoubleConvertTo.java PRE-CREATION
contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java 803f520
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/HBaseTestsSuite.java a77baba
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java ca4c07c
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestOrderedBytesConvertFunctions.java PRE-CREATION
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestTableGenerator.java 097947c
Diff: https://reviews.apache.org/r/36558/diff/
Testing
-------
Added a new unit-test class TestOrderedBytesConvertFunctions.java which derives from TestConvertFunctions.java class.
This tests the sanity of the OrderedBytesDoubleConvert{From|To}.java implementation.
Also, added a new test case to TestHBaseFilterPushDown.java class which checks if the we were able to prune the scan range of the table where the row-key was encoded as OrderedBytes.
All the added tests work as expected.
Thanks,
Smidth Panchamia
Re: Review Request 36558: DRILL-3492: Add support for
encoding/decoding of 'double' to/from OrderedBytes format
Posted by Aditya Kishore <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36558/#review92346
-----------------------------------------------------------
contrib/storage-hbase/pom.xml (lines 105 - 122)
<https://reviews.apache.org/r/36558/#comment146477>
We do not need all source code, just the function definitions. This should be sufficient.
````
....
<configuration>
<outputDirectory>${basedir}/target/classes/org/apache/drill/exec/expr/fn/impl</outputDirectory>
<resources>
<resource>
<directory>src/main/java/org/apache/drill/exec/expr/fn/impl</directory>
<filtering>true</filtering>
</resource>
</resources>
</configuration>
....
```
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java (line 115)
<https://reviews.apache.org/r/36558/#comment146472>
=> TestTableDoubleOB
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java (line 129)
<https://reviews.apache.org/r/36558/#comment146473>
=> TestTableDoubleOB
Please submit the updated patch in git's mailbox format.
- Aditya Kishore
On July 16, 2015, 2:07 p.m., Smidth Panchamia wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36558/
> -----------------------------------------------------------
>
> (Updated July 16, 2015, 2:07 p.m.)
>
>
> Review request for drill and Aditya Kishore.
>
>
> Bugs: DRILL-3492
> https://issues.apache.org/jira/browse/DRILL-3492
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> This change allows encoding/decoding of data from Double data type to OrderedBytes format.
> The following JIRA added the OrderedBytes encoding to HBase: https://issues.apache.org/jira/browse/HBASE-8201
>
> This encoding scheme will preserve the sort-order of the native data-type when it is stored as sorted byte arrays on disk.
> Thus, it will help the HBase storage plugin if the row-keys have been encoded in OrderedBytes format.
>
> This functionality allows us to prune the scan ranges, thus reading much lesser data from the server.
>
>
> Diffs
> -----
>
> contrib/storage-hbase/pom.xml d027771
> contrib/storage-hbase/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/OrderedBytesDoubleConvertFrom.java PRE-CREATION
> contrib/storage-hbase/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/OrderedBytesDoubleConvertTo.java PRE-CREATION
> contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java 803f520
> contrib/storage-hbase/src/test/java/org/apache/drill/hbase/HBaseTestsSuite.java a77baba
> contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java ca4c07c
> contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestOrderedBytesConvertFunctions.java PRE-CREATION
> contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestTableGenerator.java 097947c
>
> Diff: https://reviews.apache.org/r/36558/diff/
>
>
> Testing
> -------
>
> Added a new unit-test class TestOrderedBytesConvertFunctions.java which derives from TestConvertFunctions.java class.
> This tests the sanity of the OrderedBytesDoubleConvert{From|To}.java implementation.
>
> Also, added a new test case to TestHBaseFilterPushDown.java class which checks if the we were able to prune the scan range of the table where the row-key was encoded as OrderedBytes.
>
> All the added tests work as expected.
>
>
> Thanks,
>
> Smidth Panchamia
>
>
Re: Review Request 36558: DRILL-3492: Add support for
encoding/decoding of 'double' to/from OrderedBytes format
Posted by Smidth Panchamia <sp...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36558/
-----------------------------------------------------------
(Updated July 27, 2015, 6:18 p.m.)
Review request for drill and Aditya Kishore.
Changes
-------
Updated the patch as per comments and made it in mail format.
Bugs: DRILL-3492
https://issues.apache.org/jira/browse/DRILL-3492
Repository: drill-git
Description
-------
This change allows encoding/decoding of data from Double data type to OrderedBytes format.
The following JIRA added the OrderedBytes encoding to HBase: https://issues.apache.org/jira/browse/HBASE-8201
This encoding scheme will preserve the sort-order of the native data-type when it is stored as sorted byte arrays on disk.
Thus, it will help the HBase storage plugin if the row-keys have been encoded in OrderedBytes format.
This functionality allows us to prune the scan ranges, thus reading much lesser data from the server.
Diffs (updated)
-----
contrib/storage-hbase/pom.xml d027771
contrib/storage-hbase/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/OrderedBytesDoubleConvertFrom.java PRE-CREATION
contrib/storage-hbase/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/OrderedBytesDoubleConvertTo.java PRE-CREATION
contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java 803f520
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/HBaseTestsSuite.java a77baba
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java ca4c07c
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestOrderedBytesConvertFunctions.java PRE-CREATION
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestTableGenerator.java 097947c
Diff: https://reviews.apache.org/r/36558/diff/
Testing
-------
Added a new unit-test class TestOrderedBytesConvertFunctions.java which derives from TestConvertFunctions.java class.
This tests the sanity of the OrderedBytesDoubleConvert{From|To}.java implementation.
Also, added a new test case to TestHBaseFilterPushDown.java class which checks if the we were able to prune the scan range of the table where the row-key was encoded as OrderedBytes.
All the added tests work as expected.
Thanks,
Smidth Panchamia