[jira] [Created] (DRILL-7012) Make SelectionVectorRemover project only the needed columns

["Boaz Ben-Zvi (JIRA)" <ji...@apache.org>]

Boaz Ben-Zvi created DRILL-7012:
-----------------------------------

             Summary: Make SelectionVectorRemover project only the needed columns
                 Key: DRILL-7012
                 URL: https://issues.apache.org/jira/browse/DRILL-7012
             Project: Apache Drill
          Issue Type: Improvement
          Components: Execution - Relational Operators, Query Planning & Optimization
    Affects Versions: 1.15.0
            Reporter: Boaz Ben-Zvi


   A SelectionVectorRemover is often used after a filter, to copy into a newly allocated new batch only the "filtered out" rows. In some cases the columns used by the filter are not needed downstream; currently these columns are being needlessly allocated and copied, and later removed by a Project.

  _+Suggested improvement+_: The planner can pass the information about these columns to the SelectionVectorRemover, which would avoid this useless allocation and copy. The Planner would also eliminate that Project from the plan.

   Here is an example, the query:
{code:java}
select max(l_quantity) from cp.`tpch/lineitem.parquet` L where L.l_orderkey > 58999 and L.l_shipmode = 'TRUCK' group by l_linenumber ;
{code}
And the result plan (trimmed for readability), where "l_orderkey" and "l_shipmode" are removed by the Project:
{noformat}
00-00 Screen : rowType = RecordType(ANY EXPR$0): 
 00-01 Project(EXPR$0=[$0]) : rowType = RecordType(ANY EXPR$0): 
 00-02 Project(EXPR$0=[$1]) : rowType = RecordType(ANY EXPR$0): 
 00-03 HashAgg(group=[\{0}], EXPR$0=[MAX($1)]) : rowType = RecordType(ANY l_linenumber, ANY EXPR$0): 
 00-04 *Project*(l_linenumber=[$2], l_quantity=[$3]) : rowType = RecordType(ANY l_linenumber, ANY l_quantity): 
 00-05 *SelectionVectorRemover* : rowType = RecordType(ANY *l_orderkey*, ANY *l_shipmode*, ANY l_linenumber, ANY l_quantity): 
 00-06 *Filter*(condition=[AND(>($0, 58999), =($1, 'TRUCK'))]) : rowType = RecordType(ANY l_orderkey, ANY l_shipmode, ANY l_linenumber, ANY l_quantity): 
 00-07 Scan(table=[[cp, tpch/lineitem.parquet]], groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=classpath:/tpch/lineitem.parquet]], selectionRoot=classpath:/tpch/lineitem.parquet, numFiles=1, numRowGroups=1, usedMetadataFile=false, columns=[`l_orderkey`, `l_shipmode`, `l_linenumber`, `l_quantity`]]]) : rowType = RecordType(ANY l_orderkey, ANY l_shipmode, ANY l_linenumber, ANY l_quantity):
{noformat}
The implementation will not be simple, as the relevant code (e.g., GenericSV2Copier) has no idea of specific columns.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)