[jira] [Created] (DRILL-6635) PartitionLimit for Lateral/Unnest

["Sorabh Hamirwasia (JIRA)" <ji...@apache.org>]

Sorabh Hamirwasia created DRILL-6635:
----------------------------------------

             Summary: PartitionLimit for Lateral/Unnest
                 Key: DRILL-6635
                 URL: https://issues.apache.org/jira/browse/DRILL-6635
             Project: Apache Drill
          Issue Type: Task
          Components: Execution - Relational Operators
    Affects Versions: 1.14.0
            Reporter: Sorabh Hamirwasia
            Assignee: Sorabh Hamirwasia


With batch processing changes in Lateral/Unnest the limit/TopN clause within Lateral-Unnest subquery will not work as expected since it will impose limit/TopN across RowId's. We need a new mechanism to apply these operators at rowId boundary.
For now we are planning to add support for only limit and hence need to have a new operator PartitionLimit which will get the partitionColumn on which the limit should be imposed. This will currently only support queries between lateral and unnest. 
For TopN we can still achieve that using combination of Sort and Partition Limit and later we can figure out how to address it directly within TopN or is it needed at all. Since the number of rows across EMIT boundary on which SORT will operate should not be big enough and mostly be done in memory.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)