You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Robert Joseph Evans (JIRA)" <ji...@apache.org> on 2012/07/18 23:49:35 UTC
[jira] [Created] (MAPREDUCE-4458) Warn in java.library.path is used
Robert Joseph Evans created MAPREDUCE-4458:
----------------------------------------------
Summary: Warn in java.library.path is used
Key: MAPREDUCE-4458
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4458
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 0.23.3, 3.0.0, 2.2.0-alpha
Reporter: Robert Joseph Evans
Assignee: Robert Joseph Evans
If java.library.path is used on the command line for launching an MRAppMaster or an MR Task, it could conflict with how standard Hadoop/HDFS JNI libraries and dependencies are found. At a minimum the client should output a warning and ask the user to switch to LD_LIBRARY_PATH. It would be nice to automatically do this for them but parsing the command line is scary so just a warning is probably good enough for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4458) Warn if java.library.path is used
for AM or Task
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-4458:
-------------------------------------------
Summary: Warn if java.library.path is used for AM or Task (was: Warn in java.library.path is used)
> Warn if java.library.path is used for AM or Task
> ------------------------------------------------
>
> Key: MAPREDUCE-4458
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4458
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.3, 3.0.0, 2.2.0-alpha
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
>
> If java.library.path is used on the command line for launching an MRAppMaster or an MR Task, it could conflict with how standard Hadoop/HDFS JNI libraries and dependencies are found. At a minimum the client should output a warning and ask the user to switch to LD_LIBRARY_PATH. It would be nice to automatically do this for them but parsing the command line is scary so just a warning is probably good enough for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4458) Warn if java.library.path is
used for AM or Task
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482441#comment-13482441 ]
Robert Joseph Evans commented on MAPREDUCE-4458:
------------------------------------------------
A few things.
# the log message for the AM does not have a space in between LD_LIBRARY_PATH and mrAppOptions. I am not even really sure why we are outputting mrAppOptions.
# The admin and the normal opts for map and reduce tasks are smashed together without any space in between. Even though this should not really be a problem in practice it would be nice to add it in for correctness sake.
# It seems like a lot of the warning could be combined into a single static method {code}private static void warnForJavaLibPath(String opts, String component, String envConf) {
if (opts != null && opts.contains("-Djava.library.path")) {
LOG.warn("Overriding java.library.path for " + component + ...);
}
{code}
# It might be good to not just have LD_LIBRARY_PATH, but also point them to the config that can be used to set LD_LIBRARY_PATH.
> Warn if java.library.path is used for AM or Task
> ------------------------------------------------
>
> Key: MAPREDUCE-4458
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4458
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.3, 3.0.0, 2.0.2-alpha
> Reporter: Robert Joseph Evans
> Assignee: Robert Parker
> Attachments: MAPREDUCE-4458.patch
>
>
> If java.library.path is used on the command line for launching an MRAppMaster or an MR Task, it could conflict with how standard Hadoop/HDFS JNI libraries and dependencies are found. At a minimum the client should output a warning and ask the user to switch to LD_LIBRARY_PATH. It would be nice to automatically do this for them but parsing the command line is scary so just a warning is probably good enough for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4458) Warn if java.library.path is used
for AM or Task
Posted by "Robert Parker (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Parker updated MAPREDUCE-4458:
-------------------------------------
Status: Open (was: Patch Available)
> Warn if java.library.path is used for AM or Task
> ------------------------------------------------
>
> Key: MAPREDUCE-4458
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4458
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 2.0.2-alpha, 0.23.3, 3.0.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Parker
> Attachments: MAPREDUCE-4458.patch
>
>
> If java.library.path is used on the command line for launching an MRAppMaster or an MR Task, it could conflict with how standard Hadoop/HDFS JNI libraries and dependencies are found. At a minimum the client should output a warning and ask the user to switch to LD_LIBRARY_PATH. It would be nice to automatically do this for them but parsing the command line is scary so just a warning is probably good enough for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-4458) Warn if java.library.path is
used for AM or Task
Posted by "Robert Parker (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Parker reassigned MAPREDUCE-4458:
----------------------------------------
Assignee: Robert Parker (was: Ravi Prakash)
> Warn if java.library.path is used for AM or Task
> ------------------------------------------------
>
> Key: MAPREDUCE-4458
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4458
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.3, 3.0.0, 2.0.2-alpha
> Reporter: Robert Joseph Evans
> Assignee: Robert Parker
>
> If java.library.path is used on the command line for launching an MRAppMaster or an MR Task, it could conflict with how standard Hadoop/HDFS JNI libraries and dependencies are found. At a minimum the client should output a warning and ask the user to switch to LD_LIBRARY_PATH. It would be nice to automatically do this for them but parsing the command line is scary so just a warning is probably good enough for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4458) Warn if java.library.path is used
for AM or Task
Posted by "Robert Parker (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Parker updated MAPREDUCE-4458:
-------------------------------------
Attachment: MAPREDUCE-4458.patch
The client will generate a warning if the java.library.path is defined in the java opts for the MRAppMaster, or the associated map or reduce task.
> Warn if java.library.path is used for AM or Task
> ------------------------------------------------
>
> Key: MAPREDUCE-4458
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4458
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.3, 3.0.0, 2.0.2-alpha
> Reporter: Robert Joseph Evans
> Assignee: Robert Parker
> Attachments: MAPREDUCE-4458.patch
>
>
> If java.library.path is used on the command line for launching an MRAppMaster or an MR Task, it could conflict with how standard Hadoop/HDFS JNI libraries and dependencies are found. At a minimum the client should output a warning and ask the user to switch to LD_LIBRARY_PATH. It would be nice to automatically do this for them but parsing the command line is scary so just a warning is probably good enough for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-4458) Warn if java.library.path is
used for AM or Task
Posted by "Ravi Prakash (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Prakash reassigned MAPREDUCE-4458:
---------------------------------------
Assignee: Ravi Prakash (was: Robert Joseph Evans)
> Warn if java.library.path is used for AM or Task
> ------------------------------------------------
>
> Key: MAPREDUCE-4458
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4458
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.3, 3.0.0, 2.0.2-alpha
> Reporter: Robert Joseph Evans
> Assignee: Ravi Prakash
>
> If java.library.path is used on the command line for launching an MRAppMaster or an MR Task, it could conflict with how standard Hadoop/HDFS JNI libraries and dependencies are found. At a minimum the client should output a warning and ask the user to switch to LD_LIBRARY_PATH. It would be nice to automatically do this for them but parsing the command line is scary so just a warning is probably good enough for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4458) Warn if java.library.path is used
for AM or Task
Posted by "Robert Parker (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Parker updated MAPREDUCE-4458:
-------------------------------------
Status: Patch Available (was: Open)
> Warn if java.library.path is used for AM or Task
> ------------------------------------------------
>
> Key: MAPREDUCE-4458
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4458
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 2.0.2-alpha, 0.23.3, 3.0.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Parker
> Attachments: MAPREDUCE-4458.patch
>
>
> If java.library.path is used on the command line for launching an MRAppMaster or an MR Task, it could conflict with how standard Hadoop/HDFS JNI libraries and dependencies are found. At a minimum the client should output a warning and ask the user to switch to LD_LIBRARY_PATH. It would be nice to automatically do this for them but parsing the command line is scary so just a warning is probably good enough for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4458) Warn if java.library.path is
used for AM or Task
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481997#comment-13481997 ]
Hadoop QA commented on MAPREDUCE-4458:
--------------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12550362/MAPREDUCE-4458.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2960//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2960//console
This message is automatically generated.
> Warn if java.library.path is used for AM or Task
> ------------------------------------------------
>
> Key: MAPREDUCE-4458
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4458
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.3, 3.0.0, 2.0.2-alpha
> Reporter: Robert Joseph Evans
> Assignee: Robert Parker
> Attachments: MAPREDUCE-4458.patch
>
>
> If java.library.path is used on the command line for launching an MRAppMaster or an MR Task, it could conflict with how standard Hadoop/HDFS JNI libraries and dependencies are found. At a minimum the client should output a warning and ask the user to switch to LD_LIBRARY_PATH. It would be nice to automatically do this for them but parsing the command line is scary so just a warning is probably good enough for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira