You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Eric Tschetter (JIRA)" <ji...@apache.org> on 2011/02/03 18:33:29 UTC
[jira] Created: (PIG-1841) TupleSize implemented incorrectly
TupleSize implemented incorrectly
---------------------------------
Key: PIG-1841
URL: https://issues.apache.org/jira/browse/PIG-1841
Project: Pig
Issue Type: Bug
Reporter: Eric Tschetter
I sent this to the list:
I'm looking at Pig's TupleSize implementation and wondering if it's
implemented correctly:
@Override
public Long exec(Tuple input) throws IOException {
try{
if (input == null) return null;
return Long.valueOf(input.size());
}catch(Exception e){
int errCode = 2106;
String msg = "Error while computing size in " +
this.getClass().getSimpleName();
throw new ExecException(msg, errCode, PigException.BUG,
e);
}
}
I have a script that looks like
A = FOREACH A GENERATE STRSPLIT(value, '\u0001') AS values;
B = FOREACH B GENERATE values, SIZE(values) AS cnt;
and cnt always ends up as 1. From the code, it looks like TupleSize
is intended to only return the number of arguments into the SIZE()
UDF? Is that really the intention and I'm using the SIZE() UDF wrong?
Or, is it just a bug and it's supposed to be written as "return
Long.valueOf(((Tuple) input.get(0)).size()))"?
I got this response back:
This is definitely a bug. Can you open a Jira ticket?
Done!
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (PIG-1841) TupleSize implemented incorrectly
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1841:
----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
All tests pass. Patch committed to both trunk and 0.8 branch. Thanks Laukik!
> TupleSize implemented incorrectly
> ---------------------------------
>
> Key: PIG-1841
> URL: https://issues.apache.org/jira/browse/PIG-1841
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Eric Tschetter
> Assignee: Laukik Chitnis
> Fix For: 0.9.0, 0.8.0
>
> Attachments: PIG-1841.patch
>
>
> I sent this to the list:
> I'm looking at Pig's TupleSize implementation and wondering if it's
> implemented correctly:
> @Override
> public Long exec(Tuple input) throws IOException {
> try{
> if (input == null) return null;
> return Long.valueOf(input.size());
> }catch(Exception e){
> int errCode = 2106;
> String msg = "Error while computing size in " +
> this.getClass().getSimpleName();
> throw new ExecException(msg, errCode, PigException.BUG,
> e);
> }
> }
> I have a script that looks like
> A = FOREACH A GENERATE STRSPLIT(value, '\u0001') AS values;
> B = FOREACH B GENERATE values, SIZE(values) AS cnt;
> and cnt always ends up as 1. From the code, it looks like TupleSize
> is intended to only return the number of arguments into the SIZE()
> UDF? Is that really the intention and I'm using the SIZE() UDF wrong?
> Or, is it just a bug and it's supposed to be written as "return
> Long.valueOf(((Tuple) input.get(0)).size()))"?
> I got this response back:
> This is definitely a bug. Can you open a Jira ticket?
> Done!
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (PIG-1841) TupleSize implemented incorrectly
Posted by "Laukik Chitnis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Laukik Chitnis updated PIG-1841:
--------------------------------
Assignee: Daniel Dai (was: Laukik Chitnis)
Status: Patch Available (was: Open)
> TupleSize implemented incorrectly
> ---------------------------------
>
> Key: PIG-1841
> URL: https://issues.apache.org/jira/browse/PIG-1841
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Eric Tschetter
> Assignee: Daniel Dai
> Fix For: 0.9.0, 0.8.0
>
> Attachments: PIG-1841.patch
>
>
> I sent this to the list:
> I'm looking at Pig's TupleSize implementation and wondering if it's
> implemented correctly:
> @Override
> public Long exec(Tuple input) throws IOException {
> try{
> if (input == null) return null;
> return Long.valueOf(input.size());
> }catch(Exception e){
> int errCode = 2106;
> String msg = "Error while computing size in " +
> this.getClass().getSimpleName();
> throw new ExecException(msg, errCode, PigException.BUG,
> e);
> }
> }
> I have a script that looks like
> A = FOREACH A GENERATE STRSPLIT(value, '\u0001') AS values;
> B = FOREACH B GENERATE values, SIZE(values) AS cnt;
> and cnt always ends up as 1. From the code, it looks like TupleSize
> is intended to only return the number of arguments into the SIZE()
> UDF? Is that really the intention and I'm using the SIZE() UDF wrong?
> Or, is it just a bug and it's supposed to be written as "return
> Long.valueOf(((Tuple) input.get(0)).size()))"?
> I got this response back:
> This is definitely a bug. Can you open a Jira ticket?
> Done!
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (PIG-1841) TupleSize implemented incorrectly
Posted by "Laukik Chitnis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Laukik Chitnis updated PIG-1841:
--------------------------------
Attachment: PIG-1841.patch
The TupleSize.exec(Tuple input) method needed to unwrap the actual Tuple whose size is to be determined from the input Tuple. Fixed the unit tests which were assuming that there would be no wrapping of the object of interest. Also added a new end-to-end test case.
Results from the test-patch run:
{noformat}
[exec]
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
{noformat}
> TupleSize implemented incorrectly
> ---------------------------------
>
> Key: PIG-1841
> URL: https://issues.apache.org/jira/browse/PIG-1841
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Eric Tschetter
> Assignee: Laukik Chitnis
> Fix For: 0.8.0, 0.9.0
>
> Attachments: PIG-1841.patch
>
>
> I sent this to the list:
> I'm looking at Pig's TupleSize implementation and wondering if it's
> implemented correctly:
> @Override
> public Long exec(Tuple input) throws IOException {
> try{
> if (input == null) return null;
> return Long.valueOf(input.size());
> }catch(Exception e){
> int errCode = 2106;
> String msg = "Error while computing size in " +
> this.getClass().getSimpleName();
> throw new ExecException(msg, errCode, PigException.BUG,
> e);
> }
> }
> I have a script that looks like
> A = FOREACH A GENERATE STRSPLIT(value, '\u0001') AS values;
> B = FOREACH B GENERATE values, SIZE(values) AS cnt;
> and cnt always ends up as 1. From the code, it looks like TupleSize
> is intended to only return the number of arguments into the SIZE()
> UDF? Is that really the intention and I'm using the SIZE() UDF wrong?
> Or, is it just a bug and it's supposed to be written as "return
> Long.valueOf(((Tuple) input.get(0)).size()))"?
> I got this response back:
> This is definitely a bug. Can you open a Jira ticket?
> Done!
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (PIG-1841) TupleSize implemented incorrectly
Posted by "Laukik Chitnis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Laukik Chitnis reassigned PIG-1841:
-----------------------------------
Assignee: Laukik Chitnis (was: Daniel Dai)
> TupleSize implemented incorrectly
> ---------------------------------
>
> Key: PIG-1841
> URL: https://issues.apache.org/jira/browse/PIG-1841
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Eric Tschetter
> Assignee: Laukik Chitnis
> Fix For: 0.8.0, 0.9.0
>
> Attachments: PIG-1841.patch
>
>
> I sent this to the list:
> I'm looking at Pig's TupleSize implementation and wondering if it's
> implemented correctly:
> @Override
> public Long exec(Tuple input) throws IOException {
> try{
> if (input == null) return null;
> return Long.valueOf(input.size());
> }catch(Exception e){
> int errCode = 2106;
> String msg = "Error while computing size in " +
> this.getClass().getSimpleName();
> throw new ExecException(msg, errCode, PigException.BUG,
> e);
> }
> }
> I have a script that looks like
> A = FOREACH A GENERATE STRSPLIT(value, '\u0001') AS values;
> B = FOREACH B GENERATE values, SIZE(values) AS cnt;
> and cnt always ends up as 1. From the code, it looks like TupleSize
> is intended to only return the number of arguments into the SIZE()
> UDF? Is that really the intention and I'm using the SIZE() UDF wrong?
> Or, is it just a bug and it's supposed to be written as "return
> Long.valueOf(((Tuple) input.get(0)).size()))"?
> I got this response back:
> This is definitely a bug. Can you open a Jira ticket?
> Done!
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (PIG-1841) TupleSize implemented incorrectly
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1841:
----------------------------
Affects Version/s: 0.8.0
Fix Version/s: 0.8.0
0.9.0
Assignee: Laukik Chitnis
> TupleSize implemented incorrectly
> ---------------------------------
>
> Key: PIG-1841
> URL: https://issues.apache.org/jira/browse/PIG-1841
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Eric Tschetter
> Assignee: Laukik Chitnis
> Fix For: 0.8.0, 0.9.0
>
>
> I sent this to the list:
> I'm looking at Pig's TupleSize implementation and wondering if it's
> implemented correctly:
> @Override
> public Long exec(Tuple input) throws IOException {
> try{
> if (input == null) return null;
> return Long.valueOf(input.size());
> }catch(Exception e){
> int errCode = 2106;
> String msg = "Error while computing size in " +
> this.getClass().getSimpleName();
> throw new ExecException(msg, errCode, PigException.BUG,
> e);
> }
> }
> I have a script that looks like
> A = FOREACH A GENERATE STRSPLIT(value, '\u0001') AS values;
> B = FOREACH B GENERATE values, SIZE(values) AS cnt;
> and cnt always ends up as 1. From the code, it looks like TupleSize
> is intended to only return the number of arguments into the SIZE()
> UDF? Is that really the intention and I'm using the SIZE() UDF wrong?
> Or, is it just a bug and it's supposed to be written as "return
> Long.valueOf(((Tuple) input.get(0)).size()))"?
> I got this response back:
> This is definitely a bug. Can you open a Jira ticket?
> Done!
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Commented: (PIG-1841) TupleSize implemented incorrectly
Posted by Daniel Dai <ji...@yahoo-inc.com>.
STRSPLIT does not define outputSchema. However, it tells Pig its return type is tuple through generics, Pig will use tuple as its output schema.
Daniel
Corbin Hoenes wrote:
> Wait did he implement outputSchema? According to the wiki without providing scheme pig assumes it's a single field of type byte array.
>
> http://wiki.apache.org/pig/UDFManual
>
> Is that the problem?
>
> Sent from my iPhone
>
> On Feb 9, 2011, at 7:39 PM, "Daniel Dai (JIRA)" <ji...@apache.org> wrote:
>
>
>> [ https://issues.apache.org/jira/browse/PIG-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992838#comment-12992838 ]
>>
>> Daniel Dai commented on PIG-1841:
>> ---------------------------------
>>
>> +1
>>
>>
>>> TupleSize implemented incorrectly
>>> ---------------------------------
>>>
>>> Key: PIG-1841
>>> URL: https://issues.apache.org/jira/browse/PIG-1841
>>> Project: Pig
>>> Issue Type: Bug
>>> Affects Versions: 0.8.0
>>> Reporter: Eric Tschetter
>>> Assignee: Laukik Chitnis
>>> Fix For: 0.8.0, 0.9.0
>>>
>>> Attachments: PIG-1841.patch
>>>
>>>
>>> I sent this to the list:
>>> I'm looking at Pig's TupleSize implementation and wondering if it's
>>> implemented correctly:
>>> @Override
>>> public Long exec(Tuple input) throws IOException {
>>> try{
>>> if (input == null) return null;
>>> return Long.valueOf(input.size());
>>> }catch(Exception e){
>>> int errCode = 2106;
>>> String msg = "Error while computing size in " +
>>> this.getClass().getSimpleName();
>>> throw new ExecException(msg, errCode, PigException.BUG,
>>> e);
>>> }
>>> }
>>> I have a script that looks like
>>> A = FOREACH A GENERATE STRSPLIT(value, '\u0001') AS values;
>>> B = FOREACH B GENERATE values, SIZE(values) AS cnt;
>>> and cnt always ends up as 1. From the code, it looks like TupleSize
>>> is intended to only return the number of arguments into the SIZE()
>>> UDF? Is that really the intention and I'm using the SIZE() UDF wrong?
>>> Or, is it just a bug and it's supposed to be written as "return
>>> Long.valueOf(((Tuple) input.get(0)).size()))"?
>>> I got this response back:
>>> This is definitely a bug. Can you open a Jira ticket?
>>> Done!
>>>
>> --
>> This message is automatically generated by JIRA.
>> -
>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>
>>
>>
Re: [jira] Commented: (PIG-1841) TupleSize implemented incorrectly
Posted by Corbin Hoenes <co...@tynt.com>.
Wait did he implement outputSchema? According to the wiki without providing scheme pig assumes it's a single field of type byte array.
http://wiki.apache.org/pig/UDFManual
Is that the problem?
Sent from my iPhone
On Feb 9, 2011, at 7:39 PM, "Daniel Dai (JIRA)" <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/PIG-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992838#comment-12992838 ]
>
> Daniel Dai commented on PIG-1841:
> ---------------------------------
>
> +1
>
>> TupleSize implemented incorrectly
>> ---------------------------------
>>
>> Key: PIG-1841
>> URL: https://issues.apache.org/jira/browse/PIG-1841
>> Project: Pig
>> Issue Type: Bug
>> Affects Versions: 0.8.0
>> Reporter: Eric Tschetter
>> Assignee: Laukik Chitnis
>> Fix For: 0.8.0, 0.9.0
>>
>> Attachments: PIG-1841.patch
>>
>>
>> I sent this to the list:
>> I'm looking at Pig's TupleSize implementation and wondering if it's
>> implemented correctly:
>> @Override
>> public Long exec(Tuple input) throws IOException {
>> try{
>> if (input == null) return null;
>> return Long.valueOf(input.size());
>> }catch(Exception e){
>> int errCode = 2106;
>> String msg = "Error while computing size in " +
>> this.getClass().getSimpleName();
>> throw new ExecException(msg, errCode, PigException.BUG,
>> e);
>> }
>> }
>> I have a script that looks like
>> A = FOREACH A GENERATE STRSPLIT(value, '\u0001') AS values;
>> B = FOREACH B GENERATE values, SIZE(values) AS cnt;
>> and cnt always ends up as 1. From the code, it looks like TupleSize
>> is intended to only return the number of arguments into the SIZE()
>> UDF? Is that really the intention and I'm using the SIZE() UDF wrong?
>> Or, is it just a bug and it's supposed to be written as "return
>> Long.valueOf(((Tuple) input.get(0)).size()))"?
>> I got this response back:
>> This is definitely a bug. Can you open a Jira ticket?
>> Done!
>
> --
> This message is automatically generated by JIRA.
> -
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
[jira] Commented: (PIG-1841) TupleSize implemented incorrectly
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992838#comment-12992838 ]
Daniel Dai commented on PIG-1841:
---------------------------------
+1
> TupleSize implemented incorrectly
> ---------------------------------
>
> Key: PIG-1841
> URL: https://issues.apache.org/jira/browse/PIG-1841
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Eric Tschetter
> Assignee: Laukik Chitnis
> Fix For: 0.8.0, 0.9.0
>
> Attachments: PIG-1841.patch
>
>
> I sent this to the list:
> I'm looking at Pig's TupleSize implementation and wondering if it's
> implemented correctly:
> @Override
> public Long exec(Tuple input) throws IOException {
> try{
> if (input == null) return null;
> return Long.valueOf(input.size());
> }catch(Exception e){
> int errCode = 2106;
> String msg = "Error while computing size in " +
> this.getClass().getSimpleName();
> throw new ExecException(msg, errCode, PigException.BUG,
> e);
> }
> }
> I have a script that looks like
> A = FOREACH A GENERATE STRSPLIT(value, '\u0001') AS values;
> B = FOREACH B GENERATE values, SIZE(values) AS cnt;
> and cnt always ends up as 1. From the code, it looks like TupleSize
> is intended to only return the number of arguments into the SIZE()
> UDF? Is that really the intention and I'm using the SIZE() UDF wrong?
> Or, is it just a bug and it's supposed to be written as "return
> Long.valueOf(((Tuple) input.get(0)).size()))"?
> I got this response back:
> This is definitely a bug. Can you open a Jira ticket?
> Done!
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira