[jira] [Created] (IMPALA-15543) Use HBO stats in HdfsScanNode.getNumInputRows()

["Quanlong Huang (Jira)" <ji...@apache.org>]

Quanlong Huang created IMPALA-15543:
---------------------------------------

             Summary: Use HBO stats in HdfsScanNode.getNumInputRows()
                 Key: IMPALA-15543
                 URL: https://issues.apache.org/jira/browse/IMPALA-15543
             Project: IMPALA
          Issue Type: Improvement
            Reporter: Quanlong Huang


If numRows is missing in HMS for any selected partitions, HdfsScanNode.getNumInputRows() returns -1. This can be improved by using HBO stats of full scan with the same partition predicates. For instance, if there are HBO stats of this query
{code:sql}
select max(id) from tbl where part_col = 1;{code}
We can use its scan output cardinality as the number of scan input rows for this query
{code:sql}
select max(id) from tbl where part_col = 1 and id < 1000{code}
assuming that part_col is a partition column and id is a non-partition column.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)