You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/01/22 19:49:00 UTC
[jira] [Commented] (DRILL-6099) Drill does not push limit past
project (flatten) if it cannot be pushed into scan
[ https://issues.apache.org/jira/browse/DRILL-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16334796#comment-16334796 ]
ASF GitHub Bot commented on DRILL-6099:
---------------------------------------
Github user chunhui-shi commented on a diff in the pull request:
https://github.com/apache/drill/pull/1096#discussion_r163048747
--- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/PlannerPhase.java ---
@@ -341,7 +346,7 @@ static RuleSet getPruneScanRules(OptimizerRulesContext optimizerRulesContext) {
ParquetPruneScanRule.getFilterOnProjectParquet(optimizerRulesContext),
ParquetPruneScanRule.getFilterOnScanParquet(optimizerRulesContext),
DrillPushLimitToScanRule.LIMIT_ON_SCAN,
- DrillPushLimitToScanRule.LIMIT_ON_PROJECT
+ DrillPushLimitToScanRule.LIMIT_ON_PROJECT_SCAN
--- End diff --
Not sure if we still need "limit_on_project_scan". In theory, limit_on_project and limit_on_scan should already cover all the cases. Have you tested with "limit_on_project_scan" disabled?
> Drill does not push limit past project (flatten) if it cannot be pushed into scan
> ---------------------------------------------------------------------------------
>
> Key: DRILL-6099
> URL: https://issues.apache.org/jira/browse/DRILL-6099
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.12.0
> Reporter: Gautam Kumar Parai
> Assignee: Gautam Kumar Parai
> Priority: Major
> Fix For: 1.13.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> It would be useful to have pushdown occur past flatten(project). Here is an example to illustrate the issue:
> {{explain plan without implementation for }}{{select name, flatten(categories) as category from dfs.`/tmp/t_json_20` LIMIT 1;}}
> {{DrillScreenRel}}{{ }}
> {{ DrillLimitRel(fetch=[1])}}{{ }}
> {{ DrillProjectRel(name=[$0], category=[FLATTEN($1)])}}
> {{ DrillScanRel(table=[[dfs, /tmp/t_json_20]], groupscan=[EasyGroupScan [selectionRoot=maprfs:/tmp/t_json_20, numFiles=1, columns=[`name`, `categories`], files=[maprfs:///tmp/t_json_20/0_0_0.json]]])}}
> =================================================================
> Content of 0_0_0.json
> =================================================================
> {
> "name" : "Eric Goldberg, MD",
> "categories" : [ "Doctors", "Health & Medical" ]
> } {
> "name" : "Pine Cone Restaurant",
> "categories" : [ "Restaurants" ]
> } {
> "name" : "Deforest Family Restaurant",
> "categories" : [ "American (Traditional)", "Restaurants" ]
> } {
> "name" : "Culver's",
> "categories" : [ "Food", "Ice Cream & Frozen Yogurt", "Fast Food", "Restaurants" ]
> } {
> "name" : "Chang Jiang Chinese Kitchen",
> "categories" : [ "Chinese", "Restaurants" ]
> }
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)