[jira] [Commented] (DRILL-6318) Push down limit past flatten is incorrect

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-6318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462550#comment-16462550 ] 

ASF GitHub Bot commented on DRILL-6318:
---------------------------------------

oleg-zinovev commented on issue #1204: DRILL-6318
URL: https://github.com/apache/drill/pull/1204#issuecomment-386322683
 
 
   Rebased

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Push down limit past flatten is incorrect
> -----------------------------------------
>
>                 Key: DRILL-6318
>                 URL: https://issues.apache.org/jira/browse/DRILL-6318
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.13.0, 1.14.0
>         Environment: Ubuntu Mate 18.04
> Apache Drill 1.14.0-SNAPSHOT
>            Reporter: Oleg Zinoviev
>            Priority: Major
>         Attachments: sample.json
>
>
> Sample data: [^sample.json]
> Result of
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json`
> {code}
> is
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> |Closed Queen|
> |Innocent Volunteer|
> |Junior Wing|
> |Lame Mantis|
> |Old Master|
> |Numb Pawn|
> |Hollow Guardian|
> |Twin Hurricane|
> |Helpless Avalange|
> Let's try to get first 3 rows:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3
> {code}
> Result has only 2 rows:
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> *Reason:* Limit was pushed down below flatten and only 3 top rows from json was selected. In this 3 rows only 2nd has items in "data" field.
> Let's try to get 3 rows from the middle:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3 offset 5
> {code}
> Result is empty.
> *Reason:* Limit and offset was pushed down below flatten and only 6, 7 and 8 row from json was selected. This 3 rows contains only 3 items in "data" field. After flatten limit and offset applies second time and reject all select items.
> Error in org/apache/drill/exec/planner/logical/DrillPushLimitToScanRule.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)