You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Pradeep Kamath (JIRA)" <ji...@apache.org> on 2009/01/31 00:12:59 UTC
[jira] Created: (PIG-650) pig should look for and use the pig
specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it
does in the HOD case
pig should look for and use the pig specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it does in the HOD case
--------------------------------------------------------------------------------------------------------------------------------
Key: PIG-650
URL: https://issues.apache.org/jira/browse/PIG-650
Project: Pig
Issue Type: Bug
Affects Versions: types_branch
Reporter: Pradeep Kamath
Assignee: Pradeep Kamath
Fix For: types_branch
Currently users can create a pig-cluster-hadoop-site.xml with pig specific overrides for hadoop properties for use on the cluster. This file is searched for in the classpath and used in the HOD case but not in the non HOD case. We should also do the same in the non HOD case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-650) pig should look for and use the pig
specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it
does in the HOD case
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12680210#action_12680210 ]
Olga Natkovich commented on PIG-650:
------------------------------------
What is the order in which different properties are applied?
I think the right order should be:
(1) hadoop-site.xml for the cluster (from hod, or the one defined for static clusters)
(2) hadoop-site.xml that pig sets for the cluster
(3) user specified properties via -D switch.
Can you, please, confirm that this is what happens? Thanks.
> pig should look for and use the pig specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it does in the HOD case
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-650
> URL: https://issues.apache.org/jira/browse/PIG-650
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Santhosh Srinivasan
> Fix For: types_branch
>
> Attachments: PIG-650.patch
>
>
> Currently users can create a pig-cluster-hadoop-site.xml with pig specific overrides for hadoop properties for use on the cluster. This file is searched for in the classpath and used in the HOD case but not in the non HOD case. We should also do the same in the non HOD case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-650) pig should look for and use the pig
specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it
does in the HOD case
Posted by "Santhosh Srinivasan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Santhosh Srinivasan updated PIG-650:
------------------------------------
Attachment: PIG-650_1.patch
Attaching a new patch that undoes a previous change.
> pig should look for and use the pig specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it does in the HOD case
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-650
> URL: https://issues.apache.org/jira/browse/PIG-650
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Santhosh Srinivasan
> Fix For: types_branch
>
> Attachments: PIG-650.patch, PIG-650_1.patch
>
>
> Currently users can create a pig-cluster-hadoop-site.xml with pig specific overrides for hadoop properties for use on the cluster. This file is searched for in the classpath and used in the HOD case but not in the non HOD case. We should also do the same in the non HOD case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-650) pig should look for and use the pig
specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it
does in the HOD case
Posted by "Santhosh Srinivasan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Santhosh Srinivasan resolved PIG-650.
-------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Patch has been committed.
> pig should look for and use the pig specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it does in the HOD case
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-650
> URL: https://issues.apache.org/jira/browse/PIG-650
> Project: Pig
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Pradeep Kamath
> Assignee: Santhosh Srinivasan
> Attachments: PIG-650.patch, PIG-650_1.patch
>
>
> Currently users can create a pig-cluster-hadoop-site.xml with pig specific overrides for hadoop properties for use on the cluster. This file is searched for in the classpath and used in the HOD case but not in the non HOD case. We should also do the same in the non HOD case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-650) pig should look for and use the pig
specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it
does in the HOD case
Posted by "Santhosh Srinivasan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12680213#action_12680213 ]
Santhosh Srinivasan commented on PIG-650:
-----------------------------------------
The order in which the properties are applied are:
1. hadoop-site.xml from a static cluster or from hod
2. hadopp-site.xml from pig
3. user specified properties on the command line
In terms of one overriding the other, its in the reverse order
1. user specified properties on the command line overrides
2. hadopp-site.xml from pig which overrides
3. hadoop-site.xml from a static cluster or from hod
I noticed that I have a change in my patch which is not required, I will resubmit a patch with this change removed.
{code}
- jobClient = new JobClient(new JobConf(configuration));
+ JobConf jobConf = new JobConf(configuration);
+ jobConf.addResource("pig-cluster-hadoop-site.xml");
+ jobClient = new JobClient(jobConf);
{code}
> pig should look for and use the pig specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it does in the HOD case
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-650
> URL: https://issues.apache.org/jira/browse/PIG-650
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Santhosh Srinivasan
> Fix For: types_branch
>
> Attachments: PIG-650.patch
>
>
> Currently users can create a pig-cluster-hadoop-site.xml with pig specific overrides for hadoop properties for use on the cluster. This file is searched for in the classpath and used in the HOD case but not in the non HOD case. We should also do the same in the non HOD case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-650) pig should look for and use the pig
specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it
does in the HOD case
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12680618#action_12680618 ]
Olga Natkovich commented on PIG-650:
------------------------------------
+1, please, commit
> pig should look for and use the pig specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it does in the HOD case
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-650
> URL: https://issues.apache.org/jira/browse/PIG-650
> Project: Pig
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Pradeep Kamath
> Assignee: Santhosh Srinivasan
> Attachments: PIG-650.patch, PIG-650_1.patch
>
>
> Currently users can create a pig-cluster-hadoop-site.xml with pig specific overrides for hadoop properties for use on the cluster. This file is searched for in the classpath and used in the HOD case but not in the non HOD case. We should also do the same in the non HOD case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-650) pig should look for and use the pig
specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it
does in the HOD case
Posted by "Santhosh Srinivasan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Santhosh Srinivasan updated PIG-650:
------------------------------------
Patch Info: [Patch Available]
> pig should look for and use the pig specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it does in the HOD case
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-650
> URL: https://issues.apache.org/jira/browse/PIG-650
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Santhosh Srinivasan
> Fix For: types_branch
>
> Attachments: PIG-650.patch
>
>
> Currently users can create a pig-cluster-hadoop-site.xml with pig specific overrides for hadoop properties for use on the cluster. This file is searched for in the classpath and used in the HOD case but not in the non HOD case. We should also do the same in the non HOD case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-650) pig should look for and use the pig
specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it
does in the HOD case
Posted by "Santhosh Srinivasan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Santhosh Srinivasan updated PIG-650:
------------------------------------
Attachment: PIG-650.patch
Attached patch addresses the issue of picking up pig-hadoop_site.xml in the non-HOD case. There are no unit test cases as this involves checking if the PIG values are picked up. The testing was performed manually.
> pig should look for and use the pig specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it does in the HOD case
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-650
> URL: https://issues.apache.org/jira/browse/PIG-650
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Santhosh Srinivasan
> Fix For: types_branch
>
> Attachments: PIG-650.patch
>
>
> Currently users can create a pig-cluster-hadoop-site.xml with pig specific overrides for hadoop properties for use on the cluster. This file is searched for in the classpath and used in the HOD case but not in the non HOD case. We should also do the same in the non HOD case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-650) pig should look for and use the pig
specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it
does in the HOD case
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich reassigned PIG-650:
----------------------------------
Assignee: Santhosh Srinivasan
> pig should look for and use the pig specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it does in the HOD case
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-650
> URL: https://issues.apache.org/jira/browse/PIG-650
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Santhosh Srinivasan
> Fix For: types_branch
>
>
> Currently users can create a pig-cluster-hadoop-site.xml with pig specific overrides for hadoop properties for use on the cluster. This file is searched for in the classpath and used in the HOD case but not in the non HOD case. We should also do the same in the non HOD case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-650) pig should look for and use the pig
specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it
does in the HOD case
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-650:
-------------------------------
Assignee: (was: Pradeep Kamath)
> pig should look for and use the pig specific 'pig-cluster-hadoop-site.xml' in the non HOD case just like it does in the HOD case
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-650
> URL: https://issues.apache.org/jira/browse/PIG-650
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Fix For: types_branch
>
>
> Currently users can create a pig-cluster-hadoop-site.xml with pig specific overrides for hadoop properties for use on the cluster. This file is searched for in the classpath and used in the HOD case but not in the non HOD case. We should also do the same in the non HOD case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.