You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Anthony Hsu (JIRA)" <ji...@apache.org> on 2016/01/04 00:47:40 UTC
[jira] [Created] (PIG-4767) Partition filter not pushed down when
filter clause references variable from another load path
Anthony Hsu created PIG-4767:
--------------------------------
Summary: Partition filter not pushed down when filter clause references variable from another load path
Key: PIG-4767
URL: https://issues.apache.org/jira/browse/PIG-4767
Project: Pig
Issue Type: Bug
Affects Versions: 0.15.0
Reporter: Anthony Hsu
To reproduce:
{noformat:title=test.pig}
a = load 'a.txt';
a_group = group a all;
a_count = foreach a_group generate COUNT(a) as count;
b = load 'mytable' using org.apache.hcatalog.pig.HCatLoader();
b = filter b by datepartition == '2015-09-01-00' and foo == a_count.count;
dump b;
{noformat}
The above query ends up reading all the table partitions. If you remove the {{foo == a_count.count}} clause or replace {{a_count.count}} with a constant, then partition filtering happens properly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)