You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@pig.apache.org by "Rohini Palaniswamy (JIRA)" <ji...@apache.org> on 2015/05/14 20:32:02 UTC

[jira] [Created] (PIG-4551) Partition filter is not pushed down in case of SPLIT

Rohini Palaniswamy created PIG-4551:
---------------------------------------

             Summary: Partition filter is not pushed down in case of SPLIT
                 Key: PIG-4551
                 URL: https://issues.apache.org/jira/browse/PIG-4551
             Project: Pig
          Issue Type: Bug
    Affects Versions: 0.11.1
            Reporter: Rohini Palaniswamy


  The below query with implicit split will not push down the partition filters and will scan the whole table. 

{code}
A  = LOAD 'db1.table1'        USING org.apache.hive.hcatalog.pig.HCatLoader();
B = FILTER A BY ( ((date=='20150501' AND pk2 =='1')) and pk3 == '127' );
C  = FILTER A BY ( ((date=='20150501' AND pk2=='1') OR (date=='20150430' AND pk2=='1')) and pk3 == '127' );
{code}

The workaround now is to write two separate LOAD statements for each FILTER. We should do that behind the scenes while planning instead of user having to do that.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)