You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/02/28 23:12:00 UTC
[jira] [Work logged] (HIVE-21778) CBO: "Struct is not null" gets
evaluated as `nullable` always causing filter miss in the query
[ https://issues.apache.org/jira/browse/HIVE-21778?focusedWorklogId=395327&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-395327 ]
ASF GitHub Bot logged work on HIVE-21778:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Feb/20 23:11
Start Date: 28/Feb/20 23:11
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on pull request #928: HIVE-21778 CBO: "Struct is not null" gets evaluated as `nullable` always causing filter miss in the query
URL: https://github.com/apache/hive/pull/928
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 395327)
Remaining Estimate: 0h
Time Spent: 10m
> CBO: "Struct is not null" gets evaluated as `nullable` always causing filter miss in the query
> ----------------------------------------------------------------------------------------------
>
> Key: HIVE-21778
> URL: https://issues.apache.org/jira/browse/HIVE-21778
> Project: Hive
> Issue Type: Bug
> Components: CBO
> Affects Versions: 4.0.0, 2.3.5
> Reporter: Rajesh Balamohan
> Assignee: Vineet Garg
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21778.1.patch, HIVE-21778.2.patch, HIVE-21778.3.patch, test_null.q, test_null.q.out
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {noformat}
> drop table if exists test_struct;
> CREATE external TABLE test_struct
> (
> f1 string,
> demo_struct struct<f1:string, f2:string, f3:string>,
> datestr string
> );
> set hive.cbo.enable=true;
> explain select * from etltmp.test_struct where datestr='2019-01-01' and demo_struct is not null;
> STAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: test_struct
> filterExpr: (datestr = '2019-01-01') (type: boolean) <----- Note that demo_struct filter is not added here
> Filter Operator
> predicate: (datestr = '2019-01-01') (type: boolean)
> Select Operator
> expressions: f1 (type: string), demo_struct (type: struct<f1:string,f2:string,f3:string>), '2019-01-01' (type: string)
> outputColumnNames: _col0, _col1, _col2
> ListSink
> set hive.cbo.enable=false;
> explain select * from etltmp.test_struct where datestr='2019-01-01' and demo_struct is not null;
> STAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: test_struct
> filterExpr: ((datestr = '2019-01-01') and demo_struct is not null) (type: boolean) <----- Note that demo_struct filter is added when CBO is turned off
> Filter Operator
> predicate: ((datestr = '2019-01-01') and demo_struct is not null) (type: boolean)
> Select Operator
> expressions: f1 (type: string), demo_struct (type: struct<f1:string,f2:string,f3:string>), '2019-01-01' (type: string)
> outputColumnNames: _col0, _col1, _col2
> ListSink
> {noformat}
> In CalcitePlanner::genFilterRelNode, the following code misses to evaluate this filter.
> {noformat}
> RexNode factoredFilterExpr = RexUtil
> .pullFactors(cluster.getRexBuilder(), convertedFilterExpr);
> {noformat}
> Note that even if we add `demo_struct.f1` it would end up pushing the filter correctly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)