You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2009/07/10 20:39:15 UTC
[jira] Created: (PIG-881) Pig should ship load udfs to the backend
Pig should ship load udfs to the backend
----------------------------------------
Key: PIG-881
URL: https://issues.apache.org/jira/browse/PIG-881
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.3.0
Reporter: Daniel Dai
Fix For: 0.4.0
Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-881) Pig should ship load udfs to the backend
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-881:
---------------------------
Status: Patch Available (was: In Progress)
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch, PIG-881-2.patch, PIG-881-3.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-881) Pig should ship load udfs to the backend
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-881:
---------------------------
Attachment: PIG-881-2.patch
One unit test fail, resubmit the patch.
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch, PIG-881-2.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-881) Pig should ship load udfs to the
backend
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729892#action_12729892 ]
Hadoop QA commented on PIG-881:
-------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12413156/PIG-881-2.patch
against trunk revision 792663.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/122/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/122/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/122/console
This message is automatically generated.
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch, PIG-881-2.patch, PIG-881-3.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-881) Pig should ship load udfs to the backend
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-881:
---------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch committed. We did a manual unit test. All test pass. For the findbug warning, it is due to a dead variable. The line reference to the variable is commented out in the patch.
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch, PIG-881-2.patch, PIG-881-3.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-881) Pig should ship load udfs to the backend
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai reassigned PIG-881:
------------------------------
Assignee: Daniel Dai
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-881) Pig should ship load udfs to the
backend
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729770#action_12729770 ]
Daniel Dai commented on PIG-881:
--------------------------------
Find some problem, I will deliver patch again shortly.
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-881) Pig should ship load udfs to the
backend
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729901#action_12729901 ]
Hadoop QA commented on PIG-881:
-------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12413171/PIG-881-3.patch
against trunk revision 793147.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/123/console
This message is automatically generated.
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch, PIG-881-2.patch, PIG-881-3.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-881) Pig should ship load udfs to the backend
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-881:
---------------------------
Attachment: PIG-881-1.patch
In the patch, I also modify JarManager. One statement in JarManager prevent all classes which is load from classpath to be included. I currently remove this but I am not sure if I am doing it right.
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-881) Pig should ship load udfs to the backend
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-881:
---------------------------
Attachment: PIG-881-3.patch
Get all unit test pass.
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch, PIG-881-2.patch, PIG-881-3.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-881) Pig should ship load udfs to the backend
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-881:
---------------------------
Attachment: PIG-881-1.patch
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-881) Pig should ship load udfs to the backend
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-881:
---------------------------
Status: In Progress (was: Patch Available)
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch, PIG-881-2.patch, PIG-881-3.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-881) Pig should ship load udfs to the
backend
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729785#action_12729785 ]
Olga Natkovich commented on PIG-881:
------------------------------------
+1; the patch looks good!
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-881) Pig should ship load udfs to the backend
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-881:
---------------------------
Attachment: (was: PIG-881-1.patch)
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-881) Pig should ship load udfs to the backend
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-881:
---------------------------
Status: Patch Available (was: Open)
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch, PIG-881-2.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-881) Pig should ship load udfs to the
backend
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729887#action_12729887 ]
Olga Natkovich commented on PIG-881:
------------------------------------
+1 on the patch. Patch process seems to ve stuck again. We ran the tests manually and they passed, so please, commit the patch.
> Pig should ship load udfs to the backend
> ----------------------------------------
>
> Key: PIG-881
> URL: https://issues.apache.org/jira/browse/PIG-881
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-881-1.patch, PIG-881-2.patch, PIG-881-3.patch
>
>
> Currently, when we use load udfs, we have to use "register" statement. It is ideal that if user put udf jars in classpath, we can omit register statement, Pig can pick the udf from classpath automatically.
> However, Pig do not ship load udfs currently, the classpath approach does not work. "register" works because Pig ship that entire jar. Pig do ship eval udfs and storage udfs, we should ship load udfs as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.