You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Khurram Faraaz (JIRA)" <ji...@apache.org> on 2015/10/01 23:58:26 UTC
[jira] [Updated] (DRILL-3881) Rowkey filter does not get pushed
into Scan
[ https://issues.apache.org/jira/browse/DRILL-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Khurram Faraaz updated DRILL-3881:
----------------------------------
Fix Version/s: 1.2.0
> Rowkey filter does not get pushed into Scan
> -------------------------------------------
>
> Key: DRILL-3881
> URL: https://issues.apache.org/jira/browse/DRILL-3881
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Affects Versions: 1.2.0
> Reporter: Khurram Faraaz
> Assignee: Smidth Panchamia
> Fix For: 1.2.0
>
>
> Rowkey filter does not get pushed down into Scan
> 4 node cluster CentOS
> Drill master commit ID: b9afcf8f
> case 1) Rowkey filter does not get pushed into Scan
> {code}
> 0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE ROW_KEY = CAST('3.0838087E38' AS FLOAT);
> +------+------+
> | text | json |
> +------+------+
> | 00-00 Screen
> 00-01 Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))])
> 00-02 SelectionVectorRemover
> 00-03 Filter(condition=[=($0, CAST('3.0838087E38'):FLOAT NOT NULL)])
> 00-04 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl, startRow=null, stopRow=null, filter=null], columns=[`*`]]])
> {code}
> case 2) Rowkey filter does not get pushed into Scan
> {code}
> 0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB') = CAST('3.0838087E38' AS FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%' order by CONVERT_FROM(ROW_KEY,'FLOAT_OB') ASC;
> +------+------+
> | text | json |
> +------+------+
> | 00-00 Screen
> 00-01 Project(RK=[$0], EXPR$1=[$1])
> 00-02 SelectionVectorRemover
> 00-03 Sort(sort0=[$0], dir0=[ASC])
> 00-04 Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))])
> 00-05 SelectionVectorRemover
> 00-06 Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), CAST('3.0838087E38'):FLOAT NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 'UTF8'), '%30838087473969088%'))])
> 00-07 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl, startRow=, stopRow=, filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, ^.*\x5CQ30838087473969088\x5CE.*$)], columns=[`*`]]])
> {code}
> Same as case (2) just that ASC is missing in order by clause.
> {code}
> 0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB') = CAST('3.0838087E38' AS FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%' order by CONVERT_FROM(ROW_KEY,'FLOAT_OB');
> +------+------+
> | text | json |
> +------+------+
> | 00-00 Screen
> 00-01 Project(RK=[$0], EXPR$1=[$1])
> 00-02 SelectionVectorRemover
> 00-03 Sort(sort0=[$0], dir0=[ASC])
> 00-04 Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))])
> 00-05 SelectionVectorRemover
> 00-06 Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), CAST('3.0838087E38'):FLOAT NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 'UTF8'), '%30838087473969088%'))])
> 00-07 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl, startRow=, stopRow=, filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, ^.*\x5CQ30838087473969088\x5CE.*$)], columns=[`*`]]])
> {code}
> Snippet that creates and inserts data into HBase table.
> {code}
> public static void main(String args[]) throws IOException {
> Configuration conf = HBaseConfiguration.create();
> conf.set("hbase.zookeeper.property.clientPort","5181");
> HBaseAdmin admin = new HBaseAdmin(conf);
> if (admin.tableExists("flt_Tbl")) {
> admin.disableTable("flt_Tbl");
> admin.deleteTable("flt_Tbl");
> }
> HTableDescriptor tableDesc = new
> HTableDescriptor(TableName.valueOf("flt_Tbl"));
> tableDesc.addFamily(new HColumnDescriptor("colfam1"));
> admin.createTable(tableDesc);
> HTable table = new HTable(conf, "flt_Tbl");
> //for (float i = (float)0.5; i <= 100.00; i += 0.75) {
> for (float i = (float)1.4E-45; i <= Float.MAX_VALUE; i += Float.MAX_VALUE / 64) {
> byte[] bytes = new byte[5];
> org.apache.hadoop.hbase.util.PositionedByteRange br =
> new org.apache.hadoop.hbase.util.SimplePositionedByteRange(bytes, 0, 5);
> org.apache.hadoop.hbase.util.OrderedBytes.encodeFloat32(br, i,
> org.apache.hadoop.hbase.util.Order.DESCENDING);
> Put p = new Put(bytes);
> p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),String.format("value %05f", i).getBytes());
> table.put(p);
> }
> table.flushCommits();
> table.close();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)