You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Khurram Faraaz (JIRA)" <ji...@apache.org> on 2015/10/01 20:02:27 UTC
[jira] [Created] (DRILL-3881) Rowkey filter does not get pushed
into Scan
Khurram Faraaz created DRILL-3881:
-------------------------------------
Summary: Rowkey filter does not get pushed into Scan
Key: DRILL-3881
URL: https://issues.apache.org/jira/browse/DRILL-3881
Project: Apache Drill
Issue Type: Bug
Components: Execution - Flow
Affects Versions: 1.2.0
Reporter: Khurram Faraaz
Assignee: Smidth Panchamia
Rowkey filter does not get pushed down into Scan
4 node cluster CentOS
Drill master commit ID: b9afcf8f
case 1) Rowkey filter does not get pushed into Scan
{code}
0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE ROW_KEY = CAST('3.0838087E38' AS FLOAT);
+------+------+
| text | json |
+------+------+
| 00-00 Screen
00-01 Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))])
00-02 SelectionVectorRemover
00-03 Filter(condition=[=($0, CAST('3.0838087E38'):FLOAT NOT NULL)])
00-04 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl, startRow=null, stopRow=null, filter=null], columns=[`*`]]])
{code}
case 2) Rowkey filter does not get pushed into Scan
{code}
0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB') = CAST('3.0838087E38' AS FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%' order by CONVERT_FROM(ROW_KEY,'FLOAT_OB') ASC;
+------+------+
| text | json |
+------+------+
| 00-00 Screen
00-01 Project(RK=[$0], EXPR$1=[$1])
00-02 SelectionVectorRemover
00-03 Sort(sort0=[$0], dir0=[ASC])
00-04 Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))])
00-05 SelectionVectorRemover
00-06 Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), CAST('3.0838087E38'):FLOAT NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 'UTF8'), '%30838087473969088%'))])
00-07 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl, startRow=, stopRow=, filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, ^.*\x5CQ30838087473969088\x5CE.*$)], columns=[`*`]]])
{code}
Same as case (2) just that ASC is missing in order by clause.
{code}
0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB') = CAST('3.0838087E38' AS FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%' order by CONVERT_FROM(ROW_KEY,'FLOAT_OB');
+------+------+
| text | json |
+------+------+
| 00-00 Screen
00-01 Project(RK=[$0], EXPR$1=[$1])
00-02 SelectionVectorRemover
00-03 Sort(sort0=[$0], dir0=[ASC])
00-04 Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))])
00-05 SelectionVectorRemover
00-06 Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), CAST('3.0838087E38'):FLOAT NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 'UTF8'), '%30838087473969088%'))])
00-07 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl, startRow=, stopRow=, filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, ^.*\x5CQ30838087473969088\x5CE.*$)], columns=[`*`]]])
{code}
Snippet that creates and inserts data into HBase table.
{code}
public static void main(String args[]) throws IOException {
Configuration conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.property.clientPort","5181");
HBaseAdmin admin = new HBaseAdmin(conf);
if (admin.tableExists("flt_Tbl")) {
admin.disableTable("flt_Tbl");
admin.deleteTable("flt_Tbl");
}
HTableDescriptor tableDesc = new
HTableDescriptor(TableName.valueOf("flt_Tbl"));
tableDesc.addFamily(new HColumnDescriptor("colfam1"));
admin.createTable(tableDesc);
HTable table = new HTable(conf, "flt_Tbl");
//for (float i = (float)0.5; i <= 100.00; i += 0.75) {
for (float i = (float)1.4E-45; i <= Float.MAX_VALUE; i += Float.MAX_VALUE / 64) {
byte[] bytes = new byte[5];
org.apache.hadoop.hbase.util.PositionedByteRange br =
new org.apache.hadoop.hbase.util.SimplePositionedByteRange(bytes, 0, 5);
org.apache.hadoop.hbase.util.OrderedBytes.encodeFloat32(br, i,
org.apache.hadoop.hbase.util.Order.DESCENDING);
Put p = new Put(bytes);
p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),String.format("value %05f", i).getBytes());
table.put(p);
}
table.flushCommits();
table.close();
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)