You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Vivek Padmanabhan (JIRA)" <ji...@apache.org> on 2011/04/20 12:18:05 UTC
[jira] [Created] (PIG-2004) Incorrect input types passed on to eval
function
Incorrect input types passed on to eval function
------------------------------------------------
Key: PIG-2004
URL: https://issues.apache.org/jira/browse/PIG-2004
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Vivek Padmanabhan
Fix For: 0.9.0
The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
| Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
The same script runs properly with Pig 0.8.
{code}
A = LOAD 'myinput' as (f1,f2,f3);
B = foreach A generate f1,f2+f3/1000.0 as doub;
C = group B by f1;
D = foreach D generate (long)(MAX(B.doub)) as f4;
dump D;
{code}
myinput
-------
a 1000 12345
b 2000 23456
c 3000 34567
a 1500 54321
b 2500 65432
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2004) Incorrect input types passed on to eval
function
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-2004:
--------------------------------
Assignee: Xuefu Zhang
> Incorrect input types passed on to eval function
> ------------------------------------------------
>
> Key: PIG-2004
> URL: https://issues.apache.org/jira/browse/PIG-2004
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
>
> The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
> | Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
> The same script runs properly with Pig 0.8.
> {code}
> A = LOAD 'myinput' as (f1,f2,f3);
> B = foreach A generate f1,f2+f3/1000.0 as doub;
> C = group B by f1;
> D = foreach C generate (long)(MAX(B.doub)) as f4;
> dump D;
> {code}
> myinput
> -------
> a 1000 12345
> b 2000 23456
> c 3000 34567
> a 1500 54321
> b 2500 65432
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2004) Incorrect input types passed on to
eval function
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022385#comment-13022385 ]
Olga Natkovich commented on PIG-2004:
-------------------------------------
Thejas - can you treat this as high priority since it can cause failures in pretty basic Pig scripts
> Incorrect input types passed on to eval function
> ------------------------------------------------
>
> Key: PIG-2004
> URL: https://issues.apache.org/jira/browse/PIG-2004
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2004-0.patch
>
>
> The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
> | Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
> The same script runs properly with Pig 0.8.
> {code}
> A = LOAD 'myinput' as (f1,f2,f3);
> B = foreach A generate f1,f2+f3/1000.0 as doub;
> C = group B by f1;
> D = foreach C generate (long)(MAX(B.doub)) as f4;
> dump D;
> {code}
> myinput
> -------
> a 1000 12345
> b 2000 23456
> c 3000 34567
> a 1500 54321
> b 2500 65432
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2004) Incorrect input types passed on to eval
function
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2004:
----------------------------
Description:
The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
| Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
The same script runs properly with Pig 0.8.
{code}
A = LOAD 'myinput' as (f1,f2,f3);
B = foreach A generate f1,f2+f3/1000.0 as doub;
C = group B by f1;
D = foreach C generate (long)(MAX(B.doub)) as f4;
dump D;
{code}
myinput
-------
a 1000 12345
b 2000 23456
c 3000 34567
a 1500 54321
b 2500 65432
was:
The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
| Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
The same script runs properly with Pig 0.8.
{code}
A = LOAD 'myinput' as (f1,f2,f3);
B = foreach A generate f1,f2+f3/1000.0 as doub;
C = group B by f1;
D = foreach D generate (long)(MAX(B.doub)) as f4;
dump D;
{code}
myinput
-------
a 1000 12345
b 2000 23456
c 3000 34567
a 1500 54321
b 2500 65432
> Incorrect input types passed on to eval function
> ------------------------------------------------
>
> Key: PIG-2004
> URL: https://issues.apache.org/jira/browse/PIG-2004
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Vivek Padmanabhan
> Fix For: 0.9.0
>
>
> The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
> | Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
> The same script runs properly with Pig 0.8.
> {code}
> A = LOAD 'myinput' as (f1,f2,f3);
> B = foreach A generate f1,f2+f3/1000.0 as doub;
> C = group B by f1;
> D = foreach C generate (long)(MAX(B.doub)) as f4;
> dump D;
> {code}
> myinput
> -------
> a 1000 12345
> b 2000 23456
> c 3000 34567
> a 1500 54321
> b 2500 65432
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2004) Incorrect input types passed on to eval
function
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2004:
----------------------------
Attachment: PIG-2004-0.patch
TypeCheckVisitor does not update schema before processing LOGenerate. Attach a patch for demonstration. Assign to Thejas for further investigation.
> Incorrect input types passed on to eval function
> ------------------------------------------------
>
> Key: PIG-2004
> URL: https://issues.apache.org/jira/browse/PIG-2004
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-2004-0.patch
>
>
> The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
> | Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
> The same script runs properly with Pig 0.8.
> {code}
> A = LOAD 'myinput' as (f1,f2,f3);
> B = foreach A generate f1,f2+f3/1000.0 as doub;
> C = group B by f1;
> D = foreach C generate (long)(MAX(B.doub)) as f4;
> dump D;
> {code}
> myinput
> -------
> a 1000 12345
> b 2000 23456
> c 3000 34567
> a 1500 54321
> b 2500 65432
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2004) Incorrect input types passed on to eval
function
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thejas M Nair updated PIG-2004:
-------------------------------
Attachment: PIG-2004.1.patch
PIG-2004.1.patch
- Reset fieldschema of all expressions from TypeCheckingExpVisitor constructor, instead of doing it in each visit function.
- Reset target fieldschema in CastExpression, copied LHS fieldschema in BinCondExpression so that uid of inner schema is not re-used.
- Fixed a NPE in LogicalSchema that was seen in test cases after this issue was fixed.
> Incorrect input types passed on to eval function
> ------------------------------------------------
>
> Key: PIG-2004
> URL: https://issues.apache.org/jira/browse/PIG-2004
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2004-0.patch, PIG-2004.1.patch
>
>
> The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
> | Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
> The same script runs properly with Pig 0.8.
> {code}
> A = LOAD 'myinput' as (f1,f2,f3);
> B = foreach A generate f1,f2+f3/1000.0 as doub;
> C = group B by f1;
> D = foreach C generate (long)(MAX(B.doub)) as f4;
> dump D;
> {code}
> myinput
> -------
> a 1000 12345
> b 2000 23456
> c 3000 34567
> a 1500 54321
> b 2500 65432
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-2004) Incorrect input types passed on to
eval function
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thejas M Nair resolved PIG-2004.
--------------------------------
Resolution: Fixed
Patch committed to trunk and 0.9 branch.
> Incorrect input types passed on to eval function
> ------------------------------------------------
>
> Key: PIG-2004
> URL: https://issues.apache.org/jira/browse/PIG-2004
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2004-0.patch, PIG-2004.1.patch
>
>
> The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
> | Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
> The same script runs properly with Pig 0.8.
> {code}
> A = LOAD 'myinput' as (f1,f2,f3);
> B = foreach A generate f1,f2+f3/1000.0 as doub;
> C = group B by f1;
> D = foreach C generate (long)(MAX(B.doub)) as f4;
> dump D;
> {code}
> myinput
> -------
> a 1000 12345
> b 2000 23456
> c 3000 34567
> a 1500 54321
> b 2500 65432
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2004) Incorrect input types passed on to
eval function
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022264#comment-13022264 ]
Daniel Dai commented on PIG-2004:
---------------------------------
Seems logical plan pick the wrong MAX implementation (should pick DoubleMax)
> Incorrect input types passed on to eval function
> ------------------------------------------------
>
> Key: PIG-2004
> URL: https://issues.apache.org/jira/browse/PIG-2004
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Vivek Padmanabhan
> Fix For: 0.9.0
>
>
> The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
> | Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
> The same script runs properly with Pig 0.8.
> {code}
> A = LOAD 'myinput' as (f1,f2,f3);
> B = foreach A generate f1,f2+f3/1000.0 as doub;
> C = group B by f1;
> D = foreach C generate (long)(MAX(B.doub)) as f4;
> dump D;
> {code}
> myinput
> -------
> a 1000 12345
> b 2000 23456
> c 3000 34567
> a 1500 54321
> b 2500 65432
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2004) Incorrect input types passed on to
eval function
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025389#comment-13025389 ]
Daniel Dai commented on PIG-2004:
---------------------------------
+1
> Incorrect input types passed on to eval function
> ------------------------------------------------
>
> Key: PIG-2004
> URL: https://issues.apache.org/jira/browse/PIG-2004
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2004-0.patch, PIG-2004.1.patch
>
>
> The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
> | Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
> The same script runs properly with Pig 0.8.
> {code}
> A = LOAD 'myinput' as (f1,f2,f3);
> B = foreach A generate f1,f2+f3/1000.0 as doub;
> C = group B by f1;
> D = foreach C generate (long)(MAX(B.doub)) as f4;
> dump D;
> {code}
> myinput
> -------
> a 1000 12345
> b 2000 23456
> c 3000 34567
> a 1500 54321
> b 2500 65432
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (PIG-2004) Incorrect input types passed on to
eval function
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai reassigned PIG-2004:
-------------------------------
Assignee: Thejas M Nair (was: Xuefu Zhang)
> Incorrect input types passed on to eval function
> ------------------------------------------------
>
> Key: PIG-2004
> URL: https://issues.apache.org/jira/browse/PIG-2004
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2004-0.patch
>
>
> The below script fails by throwing a ClassCastException from the MAX udf. The udf expects the value of the bag supplied to be databyte array, but at run time the udf gets the actual type, ie Double in this case. This causes the script execution to fail with exception;
> | Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.pig.data.DataByteArray
> The same script runs properly with Pig 0.8.
> {code}
> A = LOAD 'myinput' as (f1,f2,f3);
> B = foreach A generate f1,f2+f3/1000.0 as doub;
> C = group B by f1;
> D = foreach C generate (long)(MAX(B.doub)) as f4;
> dump D;
> {code}
> myinput
> -------
> a 1000 12345
> b 2000 23456
> c 3000 34567
> a 1500 54321
> b 2500 65432
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira