You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Ashutosh Chauhan (JIRA)" <ji...@apache.org> on 2011/08/26 22:29:29 UTC
[jira] [Created] (PIG-2245) Add end to end test for tokenize
Add end to end test for tokenize
--------------------------------
Key: PIG-2245
URL: https://issues.apache.org/jira/browse/PIG-2245
Project: Pig
Issue Type: Test
Components: internal-udfs
Affects Versions: 0.10
Reporter: Ashutosh Chauhan
Assignee: Mark Roddy
Fix For: 0.10
via test harness.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2245) Add end to end test for tokenize
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092008#comment-13092008 ]
Ashutosh Chauhan commented on PIG-2245:
---------------------------------------
Some test scenarios:
{code}
a = load 'data' as (f1:chararray);
b = foreach a generate TOKENIZE(f1);
dump b;
a = load 'data' as (f1:chararray);
b = foreach a generate TOKENIZE(f1,"|");
dump b;
a = load 'data' as (f1:chararray);
b = foreach a generate TOKENIZE(f1,'|');
dump b;
a = load 'data' as (f1:chararray);
b = foreach a generate TOKENIZE($0,'|');
dump b;
a = load 'data' as (f1:chararray);
b = foreach a generate TOKENIZE(f1,'"');
dump b;
a = load 'data' as (f1:chararray);
b = foreach a generate TOKENIZE(f1,'\'');
dump b;
{code}
> Add end to end test for tokenize
> --------------------------------
>
> Key: PIG-2245
> URL: https://issues.apache.org/jira/browse/PIG-2245
> Project: Pig
> Issue Type: Test
> Components: internal-udfs
> Affects Versions: 0.10
> Reporter: Ashutosh Chauhan
> Assignee: Mark Roddy
> Fix For: 0.10
>
>
> via test harness.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2245) Add end to end test for tokenize
Posted by "Alan Gates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-2245:
----------------------------
Attachment: PIG-2245_2.patch
Updated version of the patch that addresses my earlier comments.
> Add end to end test for tokenize
> --------------------------------
>
> Key: PIG-2245
> URL: https://issues.apache.org/jira/browse/PIG-2245
> Project: Pig
> Issue Type: Test
> Components: internal-udfs
> Affects Versions: 0.10
> Reporter: Ashutosh Chauhan
> Assignee: Mark Roddy
> Attachments: PIG-2245_2.patch, pig-tokenize.diff
>
>
> via test harness.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2245) Add end to end test for tokenize
Posted by "Olga Natkovich (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-2245:
--------------------------------
Fix Version/s: (was: 0.10)
> Add end to end test for tokenize
> --------------------------------
>
> Key: PIG-2245
> URL: https://issues.apache.org/jira/browse/PIG-2245
> Project: Pig
> Issue Type: Test
> Components: internal-udfs
> Affects Versions: 0.10
> Reporter: Ashutosh Chauhan
> Assignee: Mark Roddy
>
> via test harness.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2245) Add end to end test for tokenize
Posted by "Mark Roddy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Roddy updated PIG-2245:
----------------------------
Attachment: pig-tokenize.diff
Initial pass. Code review appreciated as I am not that familiar with perl.
> Add end to end test for tokenize
> --------------------------------
>
> Key: PIG-2245
> URL: https://issues.apache.org/jira/browse/PIG-2245
> Project: Pig
> Issue Type: Test
> Components: internal-udfs
> Affects Versions: 0.10
> Reporter: Ashutosh Chauhan
> Assignee: Mark Roddy
> Attachments: pig-tokenize.diff
>
>
> via test harness.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2245) Add end to end test for tokenize
Posted by "Alan Gates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121391#comment-13121391 ]
Alan Gates commented on PIG-2245:
---------------------------------
Tests run fine. The only question I have is did you really mean to pass the second column to TOKENIZE? In the file reg1459894 the first field is a random string of characters and numbers and the second field is a random Greek letter name. This second field, which you're passing, shouldn't ever have a 9 in it, so the second test shouldn't be a no-op. Likewise the first test is a no-op since $1 will also never have a space. Based on how TOKENIZE works the 'name' field from the studenttab10k file might be a better choice since it includes spaces.
> Add end to end test for tokenize
> --------------------------------
>
> Key: PIG-2245
> URL: https://issues.apache.org/jira/browse/PIG-2245
> Project: Pig
> Issue Type: Test
> Components: internal-udfs
> Affects Versions: 0.10
> Reporter: Ashutosh Chauhan
> Assignee: Mark Roddy
> Attachments: pig-tokenize.diff
>
>
> via test harness.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-2245) Add end to end test for tokenize
Posted by "Alan Gates (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates resolved PIG-2245.
-----------------------------
Resolution: Fixed
Fix Version/s: 0.11
Patch 2 checked in.
> Add end to end test for tokenize
> --------------------------------
>
> Key: PIG-2245
> URL: https://issues.apache.org/jira/browse/PIG-2245
> Project: Pig
> Issue Type: Test
> Components: internal-udfs
> Affects Versions: 0.10
> Reporter: Ashutosh Chauhan
> Assignee: Mark Roddy
> Fix For: 0.11
>
> Attachments: PIG-2245_2.patch, pig-tokenize.diff
>
>
> via test harness.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira