You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Joel Costigliola (JIRA)" <ji...@apache.org> on 2012/08/17 11:17:38 UTC
[jira] [Created] (PIG-2881) Add SUBTRACT eval function
Joel Costigliola created PIG-2881:
-------------------------------------
Summary: Add SUBTRACT eval function
Key: PIG-2881
URL: https://issues.apache.org/jira/browse/PIG-2881
Project: Pig
Issue Type: New Feature
Components: piggybank
Affects Versions: 0.10.0
Reporter: Joel Costigliola
Priority: Minor
Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486519#comment-13486519 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
No I did not set umask 0022 because it was not mentioned to do so in the contributor guide : https://cwiki.apache.org/PIG/howtocontribute.html.
TestStore failures :
{noformat}
Testcase: testStoreRemoteRel took 0.028 sec
FAILED
expected:<...> but was:<hdfs://joe-desktop:43074...>
...
Testcase: testStoreRemoteRelScheme took 0.04 sec
FAILED
expected:<...> but was:<hdfs://joe-desktop:43074...>
{noformat}
TestLoad failures :
{noformat}
Testcase: testLoadRemoteRel took 0.333 sec
FAILED
expected:<[hdfs://joe-desktop:49682]/tmp/test> but was:<[]/tmp/test>
junit.framework.AssertionFailedError: expected:<[hdfs://joe-desktop:49682]/tmp/test> but was:<[]/tmp/test>
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:332)
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:300)
at org.apache.pig.test.TestLoad.testLoadRemoteRel(TestLoad.java:122)
Testcase: testLoadRemoteAbs took 0.074 sec
Testcase: testLoadRemoteRelScheme took 0.03 sec
FAILED
expected:<[hdfs://joe-desktop:49682]/tmp/test> but was:<[]/tmp/test>
junit.framework.AssertionFailedError: expected:<[hdfs://joe-desktop:49682]/tmp/test> but was:<[]/tmp/test>
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:332)
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:300)
at org.apache.pig.test.TestLoad.testLoadRemoteRelScheme(TestLoad.java:139)
Testcase: testLoadRemoteAbsScheme took 27.36 sec
Testcase: testLoadRemoteAbsAuth took 0.083 sec
FAILED
expected:<[hdfs://joe-desktop:49682]/test> but was:<[]/test>
junit.framework.AssertionFailedError: expected:<[hdfs://joe-desktop:49682]/test> but was:<[]/test>
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:332)
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:300)
at org.apache.pig.test.TestLoad.testLoadRemoteAbsAuth(TestLoad.java:158)
Testcase: testLoadRemoteNormalize took 0.041 sec
Testcase: testGlobChars took 0.035 sec
FAILED
expected:<[hdfs://joe-desktop:49682]/tmp/t?s*> but was:<[]/tmp/t?s*>
junit.framework.AssertionFailedError: expected:<[hdfs://joe-desktop:49682]/tmp/t?s*> but was:<[]/tmp/t?s*>
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:332)
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:300)
at org.apache.pig.test.TestLoad.testGlobChars(TestLoad.java:174)
Testcase: testCommaSeparatedString took 0.031 sec
FAILED
expected:<[hdfs://joe-desktop:49682/tmp/usr/pig/a,hdfs://joe-desktop:49682]/tmp/usr/pig/b> but was:<[/tmp/usr/pig/a,]/tmp/usr/pig/b>
junit.framework.AssertionFailedError: expected:<[hdfs://joe-desktop:49682/tmp/usr/pig/a,hdfs://joe-desktop:49682]/tmp/usr/pig/b> but was:<[/tmp/usr/pig/a,]/tmp/usr/pig/b>
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:332)
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:300)
at org.apache.pig.test.TestLoad.testCommaSeparatedString(TestLoad.java:182)
Testcase: testCommaSeparatedString2 took 0.045 sec
FAILED
expected:<[hdfs://joe-desktop:49682/tmp/t?s*,hdfs://joe-desktop:49682]/tmp/test> but was:<[/tmp/t?s*,]/tmp/test>
junit.framework.AssertionFailedError: expected:<[hdfs://joe-desktop:49682/tmp/t?s*,hdfs://joe-desktop:49682]/tmp/test> but was:<[/tmp/t?s*,]/tmp/test>
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:332)
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:300)
at org.apache.pig.test.TestLoad.testCommaSeparatedString2(TestLoad.java:190)
Testcase: testCommaSeparatedString3 took 23.653 sec
Testcase: testCommaSeparatedString4 took 0.094 sec
FAILED
expected:<[hdfs://joe-desktop:49682/tmp/usr/pig/{a,c},hdfs://joe-desktop:49682]/tmp/usr/pig/b> but was:<[/tmp/usr/pig/{a,c},]/tmp/usr/pig/b>
junit.framework.AssertionFailedError: expected:<[hdfs://joe-desktop:49682/tmp/usr/pig/{a,c},hdfs://joe-desktop:49682]/tmp/usr/pig/b> but was:<[/tmp/usr/pig/{a,c},]/tmp/usr/pig/b>
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:332)
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:300)
at org.apache.pig.test.TestLoad.testCommaSeparatedString4(TestLoad.java:218)
Testcase: testCommaSeparatedString5 took 0.035 sec
FAILED
expected:</usr/pig/{a,c},[hdfs://joe-desktop:49682]/tmp/usr/pig/b> but was:</usr/pig/{a,c},[]/tmp/usr/pig/b>
junit.framework.AssertionFailedError: expected:</usr/pig/{a,c},[hdfs://joe-desktop:49682]/tmp/usr/pig/b> but was:</usr/pig/{a,c},[]/tmp/usr/pig/b>
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:332)
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:300)
at org.apache.pig.test.TestLoad.testCommaSeparatedString5(TestLoad.java:226)
Testcase: testCommaSeparatedString6 took 0.04 sec
FAILED
expected:<[hdfs://joe-desktop:49682]/tmp/usr/pig/{a,c},/...> but was:<[]/tmp/usr/pig/{a,c},/...>
junit.framework.AssertionFailedError: expected:<[hdfs://joe-desktop:49682]/tmp/usr/pig/{a,c},/...> but was:<[]/tmp/usr/pig/{a,c},/...>
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:332)
at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:300)
at org.apache.pig.test.TestLoad.testCommaSeparatedString6(TestLoad.java:246)
Testcase: testNonDfsLocation took 0.06 sec
{noformat}
I can attach the log files if you want (note that TestStore.txt is 1.1MB large).
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2881:
----------------------------
Assignee: Joel Costigliola
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Assignee: Joel Costigliola
> Priority: Minor
> Fix For: 0.12
>
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486869#comment-13486869 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
Hi Cheolsoo,
All tests pass after the regex change, thanks.
To be more precise, I have made a minor refactoring on the TestLoad to avoid duplication in checkLoadPath, here's what I've changed :
{code}
if(noConversionExpected) {
assertEquals(p, expected);
} else {
String protocol = pc.getExecType() == ExecType.MAPREDUCE ? "hdfs" : "file";
// regex : A word character, i.e. [a-zA-Z_0-9] or '-' followed by ':' then any characters
String regex = "[\\-\\w:\\.]";
assertTrue(p.matches(".*" + protocol + "://" + regex + "*.*"));
assertEquals(p.replaceAll(protocol + "://" + regex + "*/", "/"), expected);
}
{code}
If you are ok with that, tell I'll make the patch with SUBTRACT function and those changes.
Joel
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: PIG-2881.patch
Hope it's the final patch ;-)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: TEST-org.apache.pig.test.TestStore.txt
TEST-org.apache.pig.test.TestLoad.txt
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: PIG-2881.patch
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch, Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488148#comment-13488148 ]
Cheolsoo Park commented on PIG-2881:
------------------------------------
Looks good to me. I will wait today just to see if anyone has any concerns. If I don't hear anything, I will commit it.
Thanks a lot!
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: (was: SubtractTest.java)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487049#comment-13487049 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
Yes I did fix TestStore.
I will switch assertEquals parameters and upload the patch.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486525#comment-13486525 ]
Cheolsoo Park commented on PIG-2881:
------------------------------------
Yes, please attach the logs. Thanks!
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: SubtractTest.java
Subtract.java
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: (was: PIG-2881.patch)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch, Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487019#comment-13487019 ]
Cheolsoo Park commented on PIG-2881:
------------------------------------
Hi Joel,
That looks good. Thank you very much for fixing the tests! I assume that you would fix TestStore.java as well?
One minor comments:
- Please change the assertEquals to the following:
{code}
assertEquals(expected, p.replaceAll(protocol + "://" + regex + "*/", "/"));
{code}
- I noticed that you used 2 spaces to indent code for your original patch. Please use 4 spaces.
I will review and commit it when you upload your patch.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487412#comment-13487412 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
I agree, one question though, there is no test directory under test/org/apache/pig/builtin/, should I create one ?
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch, Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-2881:
-------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Joel!
Please someone add jocosti to the contributor list. I can't assign this jira to him.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488909#comment-13488909 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
Thanks Cheolsoo, with your help, we finally made it ! :)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Assignee: Joel Costigliola
> Priority: Minor
> Fix For: 0.12
>
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: (was: TEST-org.apache.pig.test.TestStore.txt)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486674#comment-13486674 ]
Cheolsoo Park commented on PIG-2881:
------------------------------------
Hi Joel,
I think that I know what's the problem. In fact, this is an issue with the test code as I see it. Let's look at one of error messages:
{code}
expected:<[hdfs://joe-desktop:49682]/test> but was:<[]/test>
{code}
If you look at the test code in {{TestLoad.java}}, {{hdfs://joe-desktop:49682/}} is supposed to be replaced by {{/}} as follows:
{code}
p.replaceAll("hdfs://[0-9a-zA-Z:\\.]*/", "/")
{code}
But it isn't. Why? Because your hostname {{joe-desktop:49682}} includes {{-}}, so it doesn't match the reg. exp. {{[0-9a-zA-Z:\\.]*}}.
So either you can change your hostname removing {{-}}, or you can fix the reg. exp. so that your hostname can be matched. I would recommend to do the later.
Thanks!
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487415#comment-13487415 ]
Cheolsoo Park commented on PIG-2881:
------------------------------------
I am sorry that I wasn't clear. I think that we can put the test under {{./test/org/apache/pig/builtin/}}. Right now I see one file ({{TestFunctionWrapperEvalFunc.java}}) in there. Since {{o.a.pig.builtin}} is the package that we're adding a new file, that seems like the right place to me.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch, Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: (was: Subtract.java)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487241#comment-13487241 ]
Cheolsoo Park commented on PIG-2881:
------------------------------------
Hi Joel,
Thanks for uploading the patch. It looks really good. Two minor comments:
- Can you please use Unix style newline chars?
- Can you please remove trailing white spaces?
I am not supposed to modify your patch in any way, so please forgive me for asking you do extra work.
A bigger question that I had was whether we should create a new suite (TestSUBTRACTFunc) or add new test cases to the existing TestBuiltin suite. Given that TestBuiltin is near 3k line long, I like to break this test suite into smaller pieces and make your patch be the 1st step toward that goal. But I need to know what other people think. I probably ask it on the dev mailing list. Please feel free to provide your opinion.
Thanks!
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch, Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486557#comment-13486557 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
Logs attached, resulting from playing the test without umask set to any special value.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487312#comment-13487312 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
Patch updated.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch, Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442240#comment-13442240 ]
Alan Gates commented on PIG-2881:
---------------------------------
Patch looks fine, except I don't see a reason to introduce new libraries just for asserts. We can use the existing assert functionality in junit.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: (was: PIG-2881.patch)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2881:
-----------------------------------
Status: Patch Available (was: Open)
Submitting so it shows up in the patch queue
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486504#comment-13486504 ]
Cheolsoo Park commented on PIG-2881:
------------------------------------
Hi Joel, what errors do you see in the test logs for your test failures?
build/test/logs/TEST-org.apache.pig.test.TestLoad.txt
build/test/logs/TEST-org.apache.pig.test.TestStore.txt
Both tests use MiniCluster, so they may be failing due to environment issues. For example, did you set "umask 0022" in the shell where you're running tests?
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442327#comment-13442327 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
I agree with you, I should have sent tests with basic JUnit assertions.
Truth is I was just lazy to rewrite it but I have more time now, so I can send you a test version withour Fest assertions.
You tell em.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487295#comment-13487295 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
Ok, I'll do the final modifications to have a clean patch.
I would definitely split the tests, because it will be easier to find/maintain them and it will also be possible to run only the tests you are working on instead of the complete builtin suite.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch, Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: (was: SubtractTest.java)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: (was: PIG-2881.patch)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: PIG-2881.patch
Patch attached
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch, Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487984#comment-13487984 ]
Cheolsoo Park commented on PIG-2881:
------------------------------------
Hi Joel,
+1.
Thanks you so much for your patience. The patch looks great! All tests pass too.
I have a super minor comment on the doc. I think that the functions in the doc are listed in alphabetical order, but your SUBTRACT comes before IsEmpty. I guess that you added it there because SUBTRACT is similar to DIFF? Would it make more sense to follow the order? If you could upload a new patch that fixes the order, that would be great.
Thanks!
p.s. You don't have to clean old attached files. In fact, it's better to keep them since they show how this jira has developed and make it easier for others to understand history. :-)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-2881:
-------------------------------
Fix Version/s: 0.12
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Fix For: 0.12
>
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: (was: TEST-org.apache.pig.test.TestLoad.txt)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487827#comment-13487827 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
Ok, finally found how to set up memory options in forrest (by modifying forrest.build.xml directly which is not that clean).
Anyway, I have attached the new patch, tell me if it's ok.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487746#comment-13487746 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
Ok, I have moved TestSUBTRACT to ./test/org/apache/pig/builtin/
I have updated func.xml but I can't the site, forrest always throw me this OutOfMemoryError
{noformat}
[exec] * [16/28] [0/0] 10.501s 0b basic.pdf
[exec] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[exec] at org.apache.fop.layoutmgr.BlockStackingLayoutManager.wrapPositionElement(BlockStackingLayoutManager.java:1499)
[exec] at org.apache.fop.layoutmgr.BlockStackingLayoutManager.wrapPositionElements(BlockStackingLayoutManager.java:1487)
[exec] at org.apache.fop.layoutmgr.BlockStackingLayoutManager.wrapPositionElements(BlockStackingLayoutManager.java:1471)
[exec] at org.apache.fop.layoutmgr.FlowLayoutManager.getNextKnuthElements(FlowLayoutManager.java:117)
[exec] at org.apache.fop.layoutmgr.PageBreaker.getNextKnuthElements(PageBreaker.java:145)
[exec] at org.apache.fop.layoutmgr.AbstractBreaker.getNextBlockList(AbstractBreaker.java:552)
[exec] at org.apache.fop.layoutmgr.PageBreaker.getNextBlockList(PageBreaker.java:137)
[exec] at org.apache.fop.layoutmgr.AbstractBreaker.doLayout(AbstractBreaker.java:302)
[exec] at org.apache.fop.layoutmgr.AbstractBreaker.doLayout(AbstractBreaker.java:264)
[exec] at org.apache.fop.layoutmgr.PageSequenceLayoutManager.activateLayout(PageSequenceLayoutManager.java:106)
[exec] at org.apache.fop.area.AreaTreeHandler.endPageSequence(AreaTreeHandler.java:234)
[exec] at org.apache.fop.fo.pagination.PageSequence.endOfNode(PageSequence.java:123)
[exec] at org.apache.fop.fo.FOTreeBuilder$MainFOHandler.endElement(FOTreeBuilder.java:340)
[exec] at org.apache.fop.fo.FOTreeBuilder.endElement(FOTreeBuilder.java:169)
[exec] at org.apache.cocoon.xml.AbstractXMLPipe.endElement(AbstractXMLPipe.java:112)
[exec] at org.apache.cocoon.xml.AbstractXMLPipe.endElement(AbstractXMLPipe.java:112)
[exec] at org.apache.cocoon.xml.xlink.XLinkPipe.endElement(XLinkPipe.java:212)
[exec] at org.apache.cocoon.xml.AbstractXMLPipe.endElement(AbstractXMLPipe.java:112)
[exec] at org.apache.cocoon.transformation.I18nTransformer.endElement(I18nTransformer.java:1193)
[exec] at org.apache.cocoon.components.EnvironmentChanger.endElement(EnvironmentStack.java:148)
[exec] at org.apache.xml.serializer.ToXMLSAXHandler.endElement(ToXMLSAXHandler.java:263)
[exec] at org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:1401)
[exec] at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2400)
[exec] at org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:1376)
[exec] at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2400)
[exec] at org.apache.xalan.transformer.TransformerImpl.applyTemplateToNode(TransformerImpl.java:2270)
[exec] at org.apache.xalan.transformer.TransformerImpl.transformNode(TransformerImpl.java:1356)
[exec] at org.apache.xalan.transformer.TransformerImpl.run(TransformerImpl.java:3447)
[exec] at org.apache.xalan.transformer.TransformerHandlerImpl.endDocument(TransformerHandlerImpl.java:408)
[exec] at org.apache.cocoon.xml.AbstractXMLPipe.endDocument(AbstractXMLPipe.java:56)
[exec] at org.apache.cocoon.transformation.TraxTransformer.endDocument(TraxTransformer.java:586)
[exec] at org.apache.cocoon.xml.AbstractXMLPipe.endDocument(AbstractXMLPipe.java:56)
{noformat}
I haven't been able to set Xmx memory options in forrest yet, can you help me on that ?
I have to say that the doc process is painful :(
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch, Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484199#comment-13484199 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
Hi Jon,
I'll try to do that this week end.
Regards,
Joel
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: PIG-2881.patch
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486497#comment-13486497 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
Jon,
It turns out that the trunk (rev 1403547) did not pass the following tests on my machine :(
{noformat}
[junit] Running org.apache.pig.test.TestLoad
[junit] Tests run: 15, Failures: 9, Errors: 0, Time elapsed: 59.125 sec
[junit] Running org.apache.pig.test.TestStore
[junit] Tests run: 17, Failures: 2, Errors: 0, Time elapsed: 406.874 sec
{noformat}
My environment is :
- Apache Ant(TM) version 1.8.2 compiled on December 3 2011
- Java version: 1.6.0_17, vendor: Sun Microsystems Inc.
- Default locale: en_GB, platform encoding: UTF-8
- OS name: "linux", version: "3.2.0-33-generic", arch: "amd64", family: "unix"
I have improved SUBTRACT unit tests, run all the tests and did not have more failing tests than the one I mentioned.
Question : do you want me to make a patch or wait until all tests pass on my machine ?
Regards,
Joel
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487395#comment-13487395 ]
Cheolsoo Park commented on PIG-2881:
------------------------------------
Hi Joel,
OK, I asked that on the dev list. We should keep the test as a separate file, but we should change its location and change its name.
- Please move the test to {{o.a.pig.builtin.test}}
- Please change the name to {{TestSUBTRACT.java}}.
I saw your comment regarding the test name, but if we move it to {{o.a.pig.builtin.test}}, we won't have a confusion since the other {{TestSubtract.java}} is under {{o.a.pig.test}}. Agreed?
Here are few other minor comments:
1) There is a typo in your patch.
{code:title=TestSUBTRACTFunc.java}
import static org.junit.Assert.assertTrueS; // 'S' should be removed.
{code}
2) The following import statement is not used. Please remove:
{code:title=TestSUBTRACTFunc.java}
import org.junit.Assert;
{code}
3) Please update the doc. Please look for the 'Eval Function' section in {{src/docs/src/documentation/content/xdocs/func.xml}}. To build it, run:
{code}
ant docs -Dforrest.home=<FORREST_HOME>
{code}
This will generate {{./src/docs/build/site/func.html}}.
4) Lastly, it would be better if you could re-name each test case as "testBlah". I won't insist on this, so you can leave them as they are if you want to.
Thanks!
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch, Subtract.java, SubtractTest.java, TEST-org.apache.pig.test.TestLoad.txt, TEST-org.apache.pig.test.TestStore.txt
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481575#comment-13481575 ]
Jonathan Coveney commented on PIG-2881:
---------------------------------------
Joel,
This looks fine, but for the final oomph, can you do the following:
- Put it in Pig w/the proper package (either org.apache.pig.builtin or the Piggybank)
- Generate a diff against trunk
This will make it so people can download and see exactly what is being applied. We like to avoid cases where many steps need to be taken to exactly replicate what is in trunk.
Thanks for contributing
Jon
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: (was: Subtract.java)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488047#comment-13488047 ]
Joel Costigliola commented on PIG-2881:
---------------------------------------
Sorry for cleaning old attached files, I won't do that next time.
I'll submit a new patch with doc functions in correct order.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: PIG-2881.patch
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: SubtractTest.java
new version of test depending only on JUnit assertions.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: (was: SubtractTest.java)
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2881) Add SUBTRACT eval function
Posted by "Joel Costigliola (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Costigliola updated PIG-2881:
----------------------------------
Attachment: SubtractTest.java
Subtract.java
The tests relies on fest assert https://github.com/alexruiz/fest-assert-2.x but can easily be rewritten with Hamcrest I guess.
> Add SUBTRACT eval function
> --------------------------
>
> Key: PIG-2881
> URL: https://issues.apache.org/jira/browse/PIG-2881
> Project: Pig
> Issue Type: New Feature
> Components: piggybank
> Affects Versions: 0.10.0
> Reporter: Joel Costigliola
> Priority: Minor
> Attachments: Subtract.java, SubtractTest.java
>
>
> Close to DIFF function but SUBTRACT(bag1, bag2) will subtract elements of bag2 from bag1.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira