You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Carl Steinbach (JIRA)" <ji...@apache.org> on 2009/11/13 02:38:39 UTC
[jira] Created: (HIVE-928) ScriptOperator does not set CLASSPATH of
spawned process.
ScriptOperator does not set CLASSPATH of spawned process.
---------------------------------------------------------
Key: HIVE-928
URL: https://issues.apache.org/jira/browse/HIVE-928
Project: Hadoop Hive
Issue Type: Bug
Reporter: Carl Steinbach
ScriptOperator does not set the CLASSPATH of the the spawned script process.
The practical implication of this is that Java JARs that are added using the "add JAR" command
will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-928) ScriptOperator does not set CLASSPATH
of spawned process.
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782082#action_12782082 ]
Namit Jain commented on HIVE-928:
---------------------------------
@Carl, are you working on modifying the patch so that only the JARs that were added by the user appear on the classpath of the child process
> ScriptOperator does not set CLASSPATH of spawned process.
> ---------------------------------------------------------
>
> Key: HIVE-928
> URL: https://issues.apache.org/jira/browse/HIVE-928
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
> Assignee: Carl Steinbach
> Attachments: HIVE-928, HIVE-928.patch
>
>
> ScriptOperator does not set the CLASSPATH of the the spawned script process.
> The practical implication of this is that Java JARs that are added using the "add JAR" command
> will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
> location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-928) ScriptOperator does not set CLASSPATH of
spawned process.
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-928:
--------------------------------
Attachment: HIVE-928.patch
> ScriptOperator does not set CLASSPATH of spawned process.
> ---------------------------------------------------------
>
> Key: HIVE-928
> URL: https://issues.apache.org/jira/browse/HIVE-928
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
> Attachments: HIVE-928, HIVE-928.patch
>
>
> ScriptOperator does not set the CLASSPATH of the the spawned script process.
> The practical implication of this is that Java JARs that are added using the "add JAR" command
> will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
> location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-928) ScriptOperator does not set CLASSPATH of
spawned process.
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-928:
--------------------------------
Attachment: HIVE-928
> ScriptOperator does not set CLASSPATH of spawned process.
> ---------------------------------------------------------
>
> Key: HIVE-928
> URL: https://issues.apache.org/jira/browse/HIVE-928
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
> Attachments: HIVE-928
>
>
> ScriptOperator does not set the CLASSPATH of the the spawned script process.
> The practical implication of this is that Java JARs that are added using the "add JAR" command
> will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
> location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-928) ScriptOperator does not set CLASSPATH
of spawned process.
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779715#action_12779715 ]
Carl Steinbach commented on HIVE-928:
-------------------------------------
Namit, are you concerned that inheriting the entire classpath of the parent process is not a good idea? I agree that it's unnecessary and can modify the patch so that only the JARs that were added by the user appear on the classpath of the child process. Let me know what you want me to do.
> ScriptOperator does not set CLASSPATH of spawned process.
> ---------------------------------------------------------
>
> Key: HIVE-928
> URL: https://issues.apache.org/jira/browse/HIVE-928
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
> Assignee: Carl Steinbach
> Attachments: HIVE-928, HIVE-928.patch
>
>
> ScriptOperator does not set the CLASSPATH of the the spawned script process.
> The practical implication of this is that Java JARs that are added using the "add JAR" command
> will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
> location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-928) ScriptOperator does not set CLASSPATH
of spawned process.
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782084#action_12782084 ]
Carl Steinbach commented on HIVE-928:
-------------------------------------
Yes, I'll post an updated patch by the end of the day. Sorry for the wait.
> ScriptOperator does not set CLASSPATH of spawned process.
> ---------------------------------------------------------
>
> Key: HIVE-928
> URL: https://issues.apache.org/jira/browse/HIVE-928
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
> Assignee: Carl Steinbach
> Attachments: HIVE-928, HIVE-928.patch
>
>
> ScriptOperator does not set the CLASSPATH of the the spawned script process.
> The practical implication of this is that Java JARs that are added using the "add JAR" command
> will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
> location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-928) ScriptOperator does not set CLASSPATH of
spawned process.
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-928:
--------------------------------
Tags: TRANSFORM CLASSPATH (was: TRANSFORM)
Status: Patch Available (was: Open)
> ScriptOperator does not set CLASSPATH of spawned process.
> ---------------------------------------------------------
>
> Key: HIVE-928
> URL: https://issues.apache.org/jira/browse/HIVE-928
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
> Attachments: HIVE-928, HIVE-928.patch
>
>
> ScriptOperator does not set the CLASSPATH of the the spawned script process.
> The practical implication of this is that Java JARs that are added using the "add JAR" command
> will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
> location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-928) ScriptOperator does not set CLASSPATH of
spawned process.
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-928:
----------------------------
Status: Open (was: Patch Available)
> ScriptOperator does not set CLASSPATH of spawned process.
> ---------------------------------------------------------
>
> Key: HIVE-928
> URL: https://issues.apache.org/jira/browse/HIVE-928
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
> Assignee: Carl Steinbach
> Attachments: HIVE-928, HIVE-928.patch
>
>
> ScriptOperator does not set the CLASSPATH of the the spawned script process.
> The practical implication of this is that Java JARs that are added using the "add JAR" command
> will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
> location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-928) ScriptOperator does not set CLASSPATH of
spawned process.
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-928:
--------------------------------
Component/s: Query Processor
> ScriptOperator does not set CLASSPATH of spawned process.
> ---------------------------------------------------------
>
> Key: HIVE-928
> URL: https://issues.apache.org/jira/browse/HIVE-928
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
>
> ScriptOperator does not set the CLASSPATH of the the spawned script process.
> The practical implication of this is that Java JARs that are added using the "add JAR" command
> will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
> location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-928) ScriptOperator does not set CLASSPATH
of spawned process.
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain reassigned HIVE-928:
-------------------------------
Assignee: Carl Steinbach
> ScriptOperator does not set CLASSPATH of spawned process.
> ---------------------------------------------------------
>
> Key: HIVE-928
> URL: https://issues.apache.org/jira/browse/HIVE-928
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
> Assignee: Carl Steinbach
> Attachments: HIVE-928, HIVE-928.patch
>
>
> ScriptOperator does not set the CLASSPATH of the the spawned script process.
> The practical implication of this is that Java JARs that are added using the "add JAR" command
> will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
> location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-928) ScriptOperator does not set CLASSPATH
of spawned process.
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779675#action_12779675 ]
Namit Jain commented on HIVE-928:
---------------------------------
What is the behavior in hadoop streaming ? Do the custom mappers inherit the classpath ?
> ScriptOperator does not set CLASSPATH of spawned process.
> ---------------------------------------------------------
>
> Key: HIVE-928
> URL: https://issues.apache.org/jira/browse/HIVE-928
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
> Assignee: Carl Steinbach
> Attachments: HIVE-928, HIVE-928.patch
>
>
> ScriptOperator does not set the CLASSPATH of the the spawned script process.
> The practical implication of this is that Java JARs that are added using the "add JAR" command
> will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
> location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-928) ScriptOperator does not set CLASSPATH
of spawned process.
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779752#action_12779752 ]
Namit Jain commented on HIVE-928:
---------------------------------
I wanted to know what does hadoop streaming do if I write a mapper in java - is the classpath passed ?
In any case, I think it is a good idea to do so, but we should only add the JARs that were added by the user.
> ScriptOperator does not set CLASSPATH of spawned process.
> ---------------------------------------------------------
>
> Key: HIVE-928
> URL: https://issues.apache.org/jira/browse/HIVE-928
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Carl Steinbach
> Assignee: Carl Steinbach
> Attachments: HIVE-928, HIVE-928.patch
>
>
> ScriptOperator does not set the CLASSPATH of the the spawned script process.
> The practical implication of this is that Java JARs that are added using the "add JAR" command
> will not be accessible to TRANSFORM/MAP/REDUCE operators unless the user can guess the
> location of the JAR archive on each node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.