You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "hc busy (JIRA)" <ji...@apache.org> on 2010/06/25 20:32:50 UTC
[jira] Created: (PIG-1465) Filter inside foreach is broken
Filter inside foreach is broken
-------------------------------
Key: PIG-1465
URL: https://issues.apache.org/jira/browse/PIG-1465
Project: Pig
Issue Type: Bug
Affects Versions: 0.7.0
Reporter: hc busy
{quote}
% cat data.txt
x,a,1,a
x,a,2,a
x,a,3,b
x,a,4,b
y,a,1,a
y,a,2,a
y,a,3,b
y,a,4,b
% cat script.pig
a = load 'data' as (ind:chararray, f1:chararray, num:int, f2:chararray);
b = group a by ind;
describe b;
f = foreach b{
all_total = SUM(a.num);
fed = filter a by (f1==f2);
some_total = (int)SUM(fed.num);
generate group as ind, all_total, some_total;
}
describe f;
dump f;
% pig -f script.pig
(x,a,1,a,,)
(x,a,2,a,,)
(x,a,3,b,,)
(x,a,4,b,,)
(y,a,1,a,,)
(y,a,2,a,,)
(y,a,3,b,,)
(y,a,4,b,,)
% cat what_I_expected
(x,10,3)
(y,10,3)
{quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1465) Filter inside foreach is broken
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12889266#action_12889266 ]
Richard Ding commented on PIG-1465:
-----------------------------------
I ran the script on trunk and wasn't able to reproduce (got the expected result). The only change I made to the script was adding using clause to the load statement:
{code}
a = load 'data.txt' using PigStorage(',') as (ind:chararray, f1:chararray, num:int, f2:chararray);
{code}
> Filter inside foreach is broken
> -------------------------------
>
> Key: PIG-1465
> URL: https://issues.apache.org/jira/browse/PIG-1465
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: hc busy
> Fix For: 0.8.0
>
>
> {quote}
> % cat data.txt
> x,a,1,a
> x,a,2,a
> x,a,3,b
> x,a,4,b
> y,a,1,a
> y,a,2,a
> y,a,3,b
> y,a,4,b
> % cat script.pig
> a = load 'data' as (ind:chararray, f1:chararray, num:int, f2:chararray);
> b = group a by ind;
> describe b;
> f = foreach b\{
> all_total = SUM(a.num);
> fed = filter a by (f1==f2);
> some_total = (int)SUM(fed.num);
> generate group as ind, all_total, some_total;
> \}
> describe f;
> dump f;
> % pig -f script.pig
> (x,a,1,a,,)
> (x,a,2,a,,)
> (x,a,3,b,,)
> (x,a,4,b,,)
> (y,a,1,a,,)
> (y,a,2,a,,)
> (y,a,3,b,,)
> (y,a,4,b,,)
> % cat what_I_expected
> (x,10,3)
> (y,10,3)
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1465) Filter inside foreach is broken
Posted by "hc busy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
hc busy updated PIG-1465:
-------------------------
Description:
{quote}
% cat data.txt
x,a,1,a
x,a,2,a
x,a,3,b
x,a,4,b
y,a,1,a
y,a,2,a
y,a,3,b
y,a,4,b
% cat script.pig
a = load 'data' as (ind:chararray, f1:chararray, num:int, f2:chararray);
b = group a by ind;
describe b;
f = foreach b\{
all_total = SUM(a.num);
fed = filter a by (f1==f2);
some_total = (int)SUM(fed.num);
generate group as ind, all_total, some_total;
\}
describe f;
dump f;
% pig -f script.pig
(x,a,1,a,,)
(x,a,2,a,,)
(x,a,3,b,,)
(x,a,4,b,,)
(y,a,1,a,,)
(y,a,2,a,,)
(y,a,3,b,,)
(y,a,4,b,,)
% cat what_I_expected
(x,10,3)
(y,10,3)
{quote}
was:
{quote}
% cat data.txt
x,a,1,a
x,a,2,a
x,a,3,b
x,a,4,b
y,a,1,a
y,a,2,a
y,a,3,b
y,a,4,b
% cat script.pig
a = load 'data' as (ind:chararray, f1:chararray, num:int, f2:chararray);
b = group a by ind;
describe b;
f = foreach b{
all_total = SUM(a.num);
fed = filter a by (f1==f2);
some_total = (int)SUM(fed.num);
generate group as ind, all_total, some_total;
}
describe f;
dump f;
% pig -f script.pig
(x,a,1,a,,)
(x,a,2,a,,)
(x,a,3,b,,)
(x,a,4,b,,)
(y,a,1,a,,)
(y,a,2,a,,)
(y,a,3,b,,)
(y,a,4,b,,)
% cat what_I_expected
(x,10,3)
(y,10,3)
{quote}
> Filter inside foreach is broken
> -------------------------------
>
> Key: PIG-1465
> URL: https://issues.apache.org/jira/browse/PIG-1465
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: hc busy
>
> {quote}
> % cat data.txt
> x,a,1,a
> x,a,2,a
> x,a,3,b
> x,a,4,b
> y,a,1,a
> y,a,2,a
> y,a,3,b
> y,a,4,b
> % cat script.pig
> a = load 'data' as (ind:chararray, f1:chararray, num:int, f2:chararray);
> b = group a by ind;
> describe b;
> f = foreach b\{
> all_total = SUM(a.num);
> fed = filter a by (f1==f2);
> some_total = (int)SUM(fed.num);
> generate group as ind, all_total, some_total;
> \}
> describe f;
> dump f;
> % pig -f script.pig
> (x,a,1,a,,)
> (x,a,2,a,,)
> (x,a,3,b,,)
> (x,a,4,b,,)
> (y,a,1,a,,)
> (y,a,2,a,,)
> (y,a,3,b,,)
> (y,a,4,b,,)
> % cat what_I_expected
> (x,10,3)
> (y,10,3)
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1465) Filter inside foreach is broken
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12889291#action_12889291 ]
Richard Ding commented on PIG-1465:
-----------------------------------
Ran again without the USING clause and got the incorrect result. Also ran on 0.7 and got the same results. So the problem is that the USING clause is missing in the load statement.
> Filter inside foreach is broken
> -------------------------------
>
> Key: PIG-1465
> URL: https://issues.apache.org/jira/browse/PIG-1465
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: hc busy
> Fix For: 0.8.0
>
>
> {quote}
> % cat data.txt
> x,a,1,a
> x,a,2,a
> x,a,3,b
> x,a,4,b
> y,a,1,a
> y,a,2,a
> y,a,3,b
> y,a,4,b
> % cat script.pig
> a = load 'data' as (ind:chararray, f1:chararray, num:int, f2:chararray);
> b = group a by ind;
> describe b;
> f = foreach b\{
> all_total = SUM(a.num);
> fed = filter a by (f1==f2);
> some_total = (int)SUM(fed.num);
> generate group as ind, all_total, some_total;
> \}
> describe f;
> dump f;
> % pig -f script.pig
> (x,a,1,a,,)
> (x,a,2,a,,)
> (x,a,3,b,,)
> (x,a,4,b,,)
> (y,a,1,a,,)
> (y,a,2,a,,)
> (y,a,3,b,,)
> (y,a,4,b,,)
> % cat what_I_expected
> (x,10,3)
> (y,10,3)
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1465) Filter inside foreach is broken
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-1465:
--------------------------------
Fix Version/s: 0.8.0
> Filter inside foreach is broken
> -------------------------------
>
> Key: PIG-1465
> URL: https://issues.apache.org/jira/browse/PIG-1465
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: hc busy
> Fix For: 0.8.0
>
>
> {quote}
> % cat data.txt
> x,a,1,a
> x,a,2,a
> x,a,3,b
> x,a,4,b
> y,a,1,a
> y,a,2,a
> y,a,3,b
> y,a,4,b
> % cat script.pig
> a = load 'data' as (ind:chararray, f1:chararray, num:int, f2:chararray);
> b = group a by ind;
> describe b;
> f = foreach b\{
> all_total = SUM(a.num);
> fed = filter a by (f1==f2);
> some_total = (int)SUM(fed.num);
> generate group as ind, all_total, some_total;
> \}
> describe f;
> dump f;
> % pig -f script.pig
> (x,a,1,a,,)
> (x,a,2,a,,)
> (x,a,3,b,,)
> (x,a,4,b,,)
> (y,a,1,a,,)
> (y,a,2,a,,)
> (y,a,3,b,,)
> (y,a,4,b,,)
> % cat what_I_expected
> (x,10,3)
> (y,10,3)
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-1465) Filter inside foreach is broken
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich resolved PIG-1465.
---------------------------------
Resolution: Invalid
> Filter inside foreach is broken
> -------------------------------
>
> Key: PIG-1465
> URL: https://issues.apache.org/jira/browse/PIG-1465
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: hc busy
> Fix For: 0.8.0
>
>
> {quote}
> % cat data.txt
> x,a,1,a
> x,a,2,a
> x,a,3,b
> x,a,4,b
> y,a,1,a
> y,a,2,a
> y,a,3,b
> y,a,4,b
> % cat script.pig
> a = load 'data' as (ind:chararray, f1:chararray, num:int, f2:chararray);
> b = group a by ind;
> describe b;
> f = foreach b\{
> all_total = SUM(a.num);
> fed = filter a by (f1==f2);
> some_total = (int)SUM(fed.num);
> generate group as ind, all_total, some_total;
> \}
> describe f;
> dump f;
> % pig -f script.pig
> (x,a,1,a,,)
> (x,a,2,a,,)
> (x,a,3,b,,)
> (x,a,4,b,,)
> (y,a,1,a,,)
> (y,a,2,a,,)
> (y,a,3,b,,)
> (y,a,4,b,,)
> % cat what_I_expected
> (x,10,3)
> (y,10,3)
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1465) Filter inside foreach is broken
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12889272#action_12889272 ]
Olga Natkovich commented on PIG-1465:
-------------------------------------
what happens without as clause?
> Filter inside foreach is broken
> -------------------------------
>
> Key: PIG-1465
> URL: https://issues.apache.org/jira/browse/PIG-1465
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: hc busy
> Fix For: 0.8.0
>
>
> {quote}
> % cat data.txt
> x,a,1,a
> x,a,2,a
> x,a,3,b
> x,a,4,b
> y,a,1,a
> y,a,2,a
> y,a,3,b
> y,a,4,b
> % cat script.pig
> a = load 'data' as (ind:chararray, f1:chararray, num:int, f2:chararray);
> b = group a by ind;
> describe b;
> f = foreach b\{
> all_total = SUM(a.num);
> fed = filter a by (f1==f2);
> some_total = (int)SUM(fed.num);
> generate group as ind, all_total, some_total;
> \}
> describe f;
> dump f;
> % pig -f script.pig
> (x,a,1,a,,)
> (x,a,2,a,,)
> (x,a,3,b,,)
> (x,a,4,b,,)
> (y,a,1,a,,)
> (y,a,2,a,,)
> (y,a,3,b,,)
> (y,a,4,b,,)
> % cat what_I_expected
> (x,10,3)
> (y,10,3)
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.