You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Olga Natkovich (JIRA)" <ji...@apache.org> on 2008/05/09 02:16:55 UTC
[jira] Created: (PIG-236) properties specified on the command line
are ignored
properties specified on the command line are ignored
----------------------------------------------------
Key: PIG-236
URL: https://issues.apache.org/jira/browse/PIG-236
Project: Pig
Issue Type: Bug
Reporter: Olga Natkovich
Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-236) properties specified on the command
line are ignored
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12596101#action_12596101 ]
Olga Natkovich commented on PIG-236:
------------------------------------
yes, pradeep and I tested all 3 of these cases together.
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-236) properties specified on the command
line are ignored
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12596098#action_12596098 ]
Alan Gates commented on PIG-236:
--------------------------------
These changes should be tested against a static cluster, against a cluster that uses HOD, and against a cluster that has been allocated with HOD. Pradeep, have you tested all three of those scenarios?
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
Attachment: pig-236.patch
The issue with the earlier patch was that the following code was moved up only into the non-HOD case in the "if-else"
{code}
configuration = ConfigurationUtil.toConfiguration(properties);
{code}
Now this call is made again after the if-else in the snippet below - the comment
explains the code in the snippet:
{code}
ds = new HDataStorage(properties);
// The above HDataStorage constructor sets DEFAULT_REPLICATION_FACTOR_KEY in properties.
// So we need to reconstruct the configuration object for the non HOD case
// In the HOD case, this is the first time the configuration object will be created
configuration = ConfigurationUtil.toConfiguration(properties);
{code}
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
Attachment: pig-236.patch
Patch file - details in comments section
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-236) properties specified on the command
line are ignored
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12596052#action_12596052 ]
Olga Natkovich commented on PIG-236:
------------------------------------
I don't think we need to pick up config from hadoop-default. That suppose to be used by the backend only.
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-236:
---------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Fix checked in at revision 655940. Tested this patch against static cluster, cluster using hod, and cluster allocated via hod.
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-236) properties specified on the command
line are ignored
Posted by "Pi Song (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595868#action_12595868 ]
Pi Song commented on PIG-236:
-----------------------------
Overall patch looks good except that I don't understand why you look up hadoop-site.xml in this location.
{noformat}
configuration.addResource("/hadoop-site.xml");
{noformat}
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
Fix Version/s: 0.1.0
Affects Version/s: 0.0.0
Status: Patch Available (was: In Progress)
In the code, there were several changes made:
1) In Main the earlier code was setting JOBTRACKER_LOCATION prematurely to "local" in the properties - this has been removed since this should be determined from hadoop-site.xml and then overriden by pig.properties - see the 3) below
2) Changed PropertiesUtil.java to also include properties from System including command line overrides
3) In HEexecutionEngine.java, for the non HOD case, the earlier code was only reading "properties" object (built from pig.properties/.pigrc and System.getProperties() which includes command line overrides). Now it first reads hadoop-site.xml and then adds on (over-writing existing keys) properties from the "properties" object. Towards this also added a utility function in ConfigurationUtil.java
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
Attachment: pig-236.patch
Using "hadoop-site.xml" in configuration.addResource(); in HExecutionEngine.init(Properties) instead of "/hadoop-site.xml"
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-236) properties specified on the command
line are ignored
Posted by "Amir Youssefi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595967#action_12595967 ]
Amir Youssefi commented on PIG-236:
-----------------------------------
I thinks it's time we have a Wiki page for different configuration files and how they override each other:
http://wiki.apache.org/pig/
or
http://wiki.apache.org/pig/FAQ
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-236) properties specified on the command
line are ignored
Posted by "Pi Song (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595445#action_12595445 ]
Pi Song commented on PIG-236:
-----------------------------
+1
This will increase usability
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (PIG-236) properties specified on the
command line are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595437#action_12595437 ]
pkamath edited comment on PIG-236 at 5/8/08 5:24 PM:
------------------------------------------------------------
Currently the code is not looking at the "System properties" which include the command line "-D" overrides and hence the issue reported in this bug. Here is the plan to address this:
First the pig.properties (and currently .pigrc - from comments in the code, there seems to be a plan to not support .pigrc in the future) file(s) will be read into the "properties" object. Then the "System properties" (from System.getProperties()) will be added into the properties object overriding any existing key/values.
A repercussion of this is that any default java or system properties (like java.io.tmpdir) that a user wants to override should be supplied in the commandline and NOT in pig.properties or .pigrc
was (Author: pkamath):
Currently the code is not looking at the "System properties" which include the command line "-D" overrides and hence the issue reported in this bug. Here is the plan to address this:
First the pig.properties (and currently .pigrc - from comments in the code, there seems to be a plan to not support .pigrc in the future) file(s) will be read into the "properties" object. Then the "System properties" (from System.getProperties()) will be added into the properties object overriding any existing
key/values.
A repercussion of this is that any default java or system properties (like java.io.tmpdir) that a user wants to override should be supplied in the commandline and NOT in pig.properties or .pigrc
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
Attachment: (was: pig-236.patch)
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
Status: Patch Available (was: Open)
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
Attachment: (was: pig-236.patch)
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-236) properties specified on the command line
are ignored
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich reassigned PIG-236:
----------------------------------
Assignee: Pradeep Kamath
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-236) properties specified on the command
line are ignored
Posted by "Pi Song (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12596054#action_12596054 ]
Pi Song commented on PIG-236:
-----------------------------
Like Pradeep mentioned, trying to read both "hadoop-default.xml" and "hadoop-site.xml" from classpath is a default behavior of Hadoop Configuration so we just have to live with it.
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-236) properties specified on the command
line are ignored
Posted by "Pi Song (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595873#action_12595873 ]
Pi Song commented on PIG-236:
-----------------------------
+1
We have two semantics here.
1) Some configuration files have to be under {pig.home}/conf
2) Some configuration files have to be under classpath.
Possibly we should also fix the shell script to add conf in the classpath as well so that we can say "put everything under conf"
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
Attachment: (was: pig-236.patch)
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
Attachment: pig-236.patch
The Configuration constructor by default handles hadoop-default.xml and hadoop-site.xml.
{code}
public Configuration() {
if (LOG.isDebugEnabled()) {
LOG.debug(StringUtils.stringifyException(new IOException("config()")));
}
resources.add("hadoop-default.xml");
resources.add("hadoop-site.xml");
}
{code}
Attaching a new patch to make use of this. The logic is the same as before, only with fewer changes to use the above fact.
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-236:
---------------------------
Status: Open (was: Patch Available)
Testing found that the fix did not work in the case where HOD was being used to allocate clusters. Pradeep will take a look at fixing this.
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
Hadoop looks for the named file in configuration.AddResource() in the classpath. So by specifying "/hadoop-site.xml" as the argument, we are letting the configuration.AddResource() pick it up from the first location in the classpath where such a file is present.
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
"/hadoop-site.xml" was confusing - so I changed it to "hadoop-site.xml" as indicated in previous comment and uploaded a new patch with this change.
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.0.0
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
> Fix For: 0.1.0
>
> Attachments: pig-236.patch
>
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work started: (PIG-236) properties specified on the command
line are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on PIG-236 started by Pradeep Kamath.
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-236) properties specified on the command line
are ignored
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-236:
-------------------------------
Currently the code is not looking at the "System properties" which include the command line "-D" overrides and hence the issue reported in this bug. Here is the plan to address this:
First the pig.properties (and currently .pigrc - from comments in the code, there seems to be a plan to not support .pigrc in the future) file(s) will be read into the "properties" object. Then the "System properties" (from System.getProperties()) will be added into the properties object overriding any existing
key/values.
A repercussion of this is that any default java or system properties (like java.io.tmpdir) that a user wants to override should be supplied in the commandline and NOT in pig.properties or .pigrc
> properties specified on the command line are ignored
> ----------------------------------------------------
>
> Key: PIG-236
> URL: https://issues.apache.org/jira/browse/PIG-236
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Pradeep Kamath
>
> Looks like code in src/org/apache/pig/impl/util/PropertiesUtil.java does not take system properties into account.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.