[jira] [Commented] (HIVE-17796) PTF in a view disables PPD

["Sergey Shelukhin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-17796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202991#comment-16202991 ] 

Sergey Shelukhin commented on HIVE-17796:
-----------------------------------------

[~ashutoshc] [~jcamachorodriguez] can you take a look? I am not sure how PTF interacts with PPD here

> PTF in a view disables PPD
> --------------------------
>
>                 Key: HIVE-17796
>                 URL: https://issues.apache.org/jira/browse/HIVE-17796
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>
> I disabled constant propagation to make logging cleaner. It is the same if it is enabled. See truncated path to alias; 
> Simple view with partition columns and filter outside of the view: PPD works.
> View with PTF and filter included in the view: PPD works.
> View with PTF and filter outside of the view: PPD breaks.
> I looked at the logs for some time, it looks like the predicate is already null in this case when passed to partition pruner; not sure why this is happening for now.
> View can also be partitioned.
> {noformat}
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.auto.convert.join=true;
> set hive.auto.convert.join.noconditionaltask=true;
> set hive.auto.convert.join.noconditionaltask.size=10000;
> set hive.metastore.aggregate.stats.cache.enabled=false;
> set hive.stats.fetch.column.stats=false;
> set hive.cbo.enable=false;
> create table dim (c2 string) partitioned by (pc1 string, pc2 string);
> create table fact (c1 string, c3 string) partitioned by (pc1 string, pc2 string);
> insert overwrite table dim partition (pc1='aaa', pc2='aaa') select key from src;
> insert overwrite table dim partition (pc1='ccc', pc2='ccc') select key from src;
> insert overwrite table dim partition (pc1='ddd', pc2='ddd') select key from src;
> insert overwrite table fact partition (pc1='aaa', pc2='aaa') select key, key from src;
> insert overwrite table fact partition (pc1='bbb', pc2='bbb') select key, key from src;
> insert overwrite table fact partition (pc1='ccc', pc2='ccc') select key, key from src;
> create view vw_ptf as select a1.*,
> (cast((row_number() over (partition by a1.pc1, a1.pc2)) as bigint) + b1.c2) as unique_key
> from fact a1 join dim b1 on a1.pc1 = b1.pc1 and a1.pc2 = b1.pc2;
> create view vw_simple as select a1.*, b1.c2
> from fact a1 join dim b1 on a1.pc1 = b1.pc1 and a1.pc2 = b1.pc2;
> create view vw_ppd as select a1.*,
> (cast((row_number() over (partition by a1.pc1, a1.pc2)) as bigint) + b1.c2) as Unique_Key
> from fact a1 join dim b1 on a1.pc1 = b1.pc1 and a1.pc2 = b1.pc2
> where a1.pc1 = 'ccc' and a1.pc2='ccc';
> set hive.optimize.constant.propagation=false;
> explain extended
> select a.* from vw_simple a WHERE 1 = 1 AND (a.pc1 = 'ccc' and a.pc2='ccc'); 
> explain extended
> select a.* from vw_ppd a WHERE 1 = 1 AND (a.pc1 = 'ccc' and a.pc2='ccc');
> explain extended
> select a.* from vw_ptf a WHERE 1 = 1 AND (a.pc1 = 'ccc' and a.pc2='ccc');
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)