[jira] [Created] (HBASE-26863) Rowkey pushdown does not work with complex conditions

["Yohei Kishimoto (Jira)" <ji...@apache.org>]

Yohei Kishimoto created HBASE-26863:
---------------------------------------

             Summary: Rowkey pushdown does not work with complex conditions
                 Key: HBASE-26863
                 URL: https://issues.apache.org/jira/browse/HBASE-26863
             Project: HBase
          Issue Type: Bug
          Components: hbase-connectors
    Affects Versions: connector-1.0.0
            Reporter: Yohei Kishimoto


When using pushdown column filter feature of hbase-spark-connector, issuing complex query containing rowkey conditions does not get expected rowkey pushdown.
{code:java}
{
  "table":{"namespace":"default", "name":"t1"},
  "rowkey":"key",
  "columns":{
    "KEY_FIELD":{"cf":"rowkey", "col":"key", "type":"string"},
    "A_FIELD":{"cf":"c", "col":"a", "type":"string"},
    "B_FIELD":{"cf":"c", "col":"b", "type":"string"}
  }
}
{code}
For example, given the catalog, a query `spark.sql("SELECT * FROM table WHERE KEY_FIELD >= 'get1' AND KEY_FIELD <= 'get3' AND A_FIELD IS NOT NULL")` gets incomplete rowkey pushdown (ScanRange:(upperBound:get3,isUpperBoundEqualTo:true,lowerBound:,isLowerBoundEqualTo:true)).

If a query is `spark.sql("SELECT * FROM table WHERE KEY_FIELD >= 'get1' AND KEY_FIELD <= 'get3'")`, we get normal rowkey pushdown (ScanRange:(upperBound:get3,isUpperBoundEqualTo:true,lowerBound:,isLowerBoundEqualTo:true)).

I found that ScanRange#getOverlapScanRange and ScanRange#mergeIntersect return incorrect results if the range from the arguments is wider than the instance (or  scanRange.getOverlapScanRange(scanRange) where scanRange1⊂scanRange2). Depending on the order of the Filters that the Spark optimization results produce, the order of the scan ranges that these methods receive could be the one that causes such a problem.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)