You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Olga Natkovich (JIRA)" <ji...@apache.org> on 2010/07/09 00:04:51 UTC
[jira] Created: (PIG-1489) Pig MapReduceLauncher does not use jars
in register statement
Pig MapReduceLauncher does not use jars in register statement
---------------------------------------------------------------
Key: PIG-1489
URL: https://issues.apache.org/jira/browse/PIG-1489
Project: Pig
Issue Type: Bug
Reporter: Olga Natkovich
Fix For: 0.8.0
If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
launching the mapreduce job and fail with ClassNotFoundException.
This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
register statement.
The effect is that the jars not only have to be in "register " statement in the script but also in the pig
classpath with the -classpath tag.
This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1489) Pig MapReduceLauncher does not use jars
in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1489:
------------------------------
Attachment: PIG-1489.patch
This patch adds registered jars to the class loader.
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1489) Pig MapReduceLauncher does not use
jars in register statement
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891856#action_12891856 ]
Thejas M Nair commented on PIG-1489:
------------------------------------
+1
You can commit after verifying that tests & checks are passing.
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch, PIG-1489.patch, PIG-1489_1.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1489) Pig MapReduceLauncher does not use
jars in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891861#action_12891861 ]
Richard Ding commented on PIG-1489:
-----------------------------------
test-patch results:
{code}
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 10 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
{code}
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch, PIG-1489.patch, PIG-1489_1.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1489) Pig MapReduceLauncher does not use jars
in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1489:
------------------------------
Status: Patch Available (was: Open)
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1489) Pig MapReduceLauncher does not use jars
in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1489:
------------------------------
Attachment: PIG-1489_1.patch
New patch adding the source code of the test jar.
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch, PIG-1489.patch, PIG-1489_1.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1489) Pig MapReduceLauncher does not use jars
in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1489:
------------------------------
Status: Open (was: Patch Available)
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch, PIG-1489.patch, PIG-1489_1.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1489) Pig MapReduceLauncher does not use
jars in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding reassigned PIG-1489:
---------------------------------
Assignee: Richard Ding
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1489) Pig MapReduceLauncher does not use jars
in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1489:
------------------------------
Status: Patch Available (was: Open)
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch, PIG-1489.patch, PIG-1489_1.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1489) Pig MapReduceLauncher does not use
jars in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892369#action_12892369 ]
Richard Ding commented on PIG-1489:
-----------------------------------
I ran core tests manually and they passed.
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch, PIG-1489.patch, PIG-1489_1.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1489) Pig MapReduceLauncher does not use jars
in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1489:
------------------------------
Attachment: PIG-1489.patch
re-sync with trunk.
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch, PIG-1489.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1489) Pig MapReduceLauncher does not use jars
in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1489:
------------------------------
Status: Open (was: Patch Available)
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1489) Pig MapReduceLauncher does not use jars
in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1489:
------------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch, PIG-1489.patch, PIG-1489_1.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1489) Pig MapReduceLauncher does not use jars
in register statement
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1489:
------------------------------
Status: Patch Available (was: Open)
> Pig MapReduceLauncher does not use jars in register statement
> ---------------------------------------------------------------
>
> Key: PIG-1489
> URL: https://issues.apache.org/jira/browse/PIG-1489
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1489.patch, PIG-1489.patch
>
>
> If my Pig StorFunc has its own OutputFormat class then Pig MapReducelauncher will try to instantiate it before
> launching the mapreduce job and fail with ClassNotFoundException.
> This happens because Pig MapReduce launcher uses its own classloader and ignores the classes in the jars in the
> register statement.
> The effect is that the jars not only have to be in "register " statement in the script but also in the pig
> classpath with the -classpath tag.
> This can be remedied by making the Pig MapReduceLauncher constructing a classloader that includes the registered jars
> and using that to instantiate the OutputFormat class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.