You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2010/03/05 20:02:27 UTC
[jira] Created: (PIG-1277) Pig should give error message when
cogroup on tuple keys of different inner type
Pig should give error message when cogroup on tuple keys of different inner type
--------------------------------------------------------------------------------
Key: PIG-1277
URL: https://issues.apache.org/jira/browse/PIG-1277
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.6.0
Reporter: Daniel Dai
When we cogroup on a tuple, if the inner type of tuple does not match, we treat them as different keys. This is confusing. It is desirable to give error/warnings when it happens.
Here is one example:
UDF:
{code}
public class MapGenerate extends EvalFunc<Map> {
@Override
public Map exec(Tuple input) throws IOException {
// TODO Auto-generated method stub
Map m = new HashMap();
m.put("key", new Integer(input.size()));
return m;
}
@Override
public Schema outputSchema(Schema input) {
return new Schema(new Schema.FieldSchema(null, DataType.MAP));
}
}
{code}
Pig script:
{code}
a = load '1.txt' as (a0);
b = foreach a generate a0, MapGenerate(*) as m:map[];
c = foreach b generate a0, m#'key' as key;
d = load '2.txt' as (c0, c1);
e = cogroup c by (a0, key), d by (c0, c1);
dump e;
{code}
1.txt
{code}
1
{code}
2.txt
{code}
1 1
{code}
User expected result (which is not right):
{code}
((1,1),{(1,1)},{(1,1)})
{code}
Real result:
{code}
((1,1),{(1,1)},{})
((1,1),{},{(1,1)})
{code}
We shall give user the message that we can not merge the key due to the type mismatch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1277) Pig should give error message when
cogroup on tuple keys of different inner type
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates reassigned PIG-1277:
-------------------------------
Assignee: Alan Gates
> Pig should give error message when cogroup on tuple keys of different inner type
> --------------------------------------------------------------------------------
>
> Key: PIG-1277
> URL: https://issues.apache.org/jira/browse/PIG-1277
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Assignee: Alan Gates
> Fix For: 0.9.0
>
>
> When we cogroup on a tuple, if the inner type of tuple does not match, we treat them as different keys. This is confusing. It is desirable to give error/warnings when it happens.
> Here is one example:
> UDF:
> {code}
> public class MapGenerate extends EvalFunc<Map> {
> @Override
> public Map exec(Tuple input) throws IOException {
> // TODO Auto-generated method stub
> Map m = new HashMap();
> m.put("key", new Integer(input.size()));
> return m;
> }
>
> @Override
> public Schema outputSchema(Schema input) {
> return new Schema(new Schema.FieldSchema(null, DataType.MAP));
> }
> }
> {code}
> Pig script:
> {code}
> a = load '1.txt' as (a0);
> b = foreach a generate a0, MapGenerate(*) as m:map[];
> c = foreach b generate a0, m#'key' as key;
> d = load '2.txt' as (c0, c1);
> e = cogroup c by (a0, key), d by (c0, c1);
> dump e;
> {code}
> 1.txt
> {code}
> 1
> {code}
> 2.txt
> {code}
> 1 1
> {code}
> User expected result (which is not right):
> {code}
> ((1,1),{(1,1)},{(1,1)})
> {code}
> Real result:
> {code}
> ((1,1),{(1,1)},{})
> ((1,1),{},{(1,1)})
> {code}
> We shall give user the message that we can not merge the key due to the type mismatch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1277) Pig should give error message when
cogroup on tuple keys of different inner type
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-1277:
--------------------------------
Fix Version/s: 0.9.0
> Pig should give error message when cogroup on tuple keys of different inner type
> --------------------------------------------------------------------------------
>
> Key: PIG-1277
> URL: https://issues.apache.org/jira/browse/PIG-1277
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.9.0
>
>
> When we cogroup on a tuple, if the inner type of tuple does not match, we treat them as different keys. This is confusing. It is desirable to give error/warnings when it happens.
> Here is one example:
> UDF:
> {code}
> public class MapGenerate extends EvalFunc<Map> {
> @Override
> public Map exec(Tuple input) throws IOException {
> // TODO Auto-generated method stub
> Map m = new HashMap();
> m.put("key", new Integer(input.size()));
> return m;
> }
>
> @Override
> public Schema outputSchema(Schema input) {
> return new Schema(new Schema.FieldSchema(null, DataType.MAP));
> }
> }
> {code}
> Pig script:
> {code}
> a = load '1.txt' as (a0);
> b = foreach a generate a0, MapGenerate(*) as m:map[];
> c = foreach b generate a0, m#'key' as key;
> d = load '2.txt' as (c0, c1);
> e = cogroup c by (a0, key), d by (c0, c1);
> dump e;
> {code}
> 1.txt
> {code}
> 1
> {code}
> 2.txt
> {code}
> 1 1
> {code}
> User expected result (which is not right):
> {code}
> ((1,1),{(1,1)},{(1,1)})
> {code}
> Real result:
> {code}
> ((1,1),{(1,1)},{})
> ((1,1),{},{(1,1)})
> {code}
> We shall give user the message that we can not merge the key due to the type mismatch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.