You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Kihwal Lee (Created) (JIRA)" <ji...@apache.org> on 2011/10/25 12:16:32 UTC
[jira] [Created] (MAPREDUCE-3259) ContainerLocalizer should get the
proper java.library.path from LinuxContainerExecutor
ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
--------------------------------------------------------------------------------------
Key: MAPREDUCE-3259
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: nodemanager
Affects Versions: 0.23.0, 0.24.0
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136912#comment-13136912 ]
Hudson commented on MAPREDUCE-3259:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1182 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1182/])
MAPREDUCE-3259. Added java.library.path of NodeManager to ContainerLocalizer in LinuxContainerExecutor. Contributed by Kihwal Lee.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189667
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137025#comment-13137025 ]
Hudson commented on MAPREDUCE-3259:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #52 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/52/])
Merge -c 1189667 from trunk to branch-0.23 to fix MAPREDUCE-3259.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189668
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137064#comment-13137064 ]
Hudson commented on MAPREDUCE-3259:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #873 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/873/])
MAPREDUCE-3259. Added java.library.path of NodeManager to ContainerLocalizer in LinuxContainerExecutor. Contributed by Kihwal Lee.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189667
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136917#comment-13136917 ]
Hudson commented on MAPREDUCE-3259:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #68 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/68/])
Merge -c 1189667 from trunk to branch-0.23 to fix MAPREDUCE-3259.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189668
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136869#comment-13136869 ]
Arun C Murthy commented on MAPREDUCE-3259:
------------------------------------------
Ok, the talk about apps confused me. Now I see the real issue: ContainerLocalizer needs this set for UGI etc.
We shouldn't change MapReduceChildJVM, the changes to LCE.startLocalizer seem fine.
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3259) ContainerLocalizer should get the
proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated MAPREDUCE-3259:
----------------------------------
Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0)
Status: Patch Available (was: Open)
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3259) ContainerLocalizer should get the
proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated MAPREDUCE-3259:
----------------------------------
Priority: Blocker (was: Critical)
Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0)
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: mapreduce-3259.patch
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3259) ContainerLocalizer should get the
proper java.library.path from LinuxContainerExecutor
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3259:
-------------------------------------
Resolution: Fixed
Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0)
Status: Resolved (was: Patch Available)
I just committed this. Thanks Kihwal!
Unit test failure was unrelated and fixed by MAPREDUCE-3279.
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134998#comment-13134998 ]
Hadoop QA commented on MAPREDUCE-3259:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12500662/mapreduce-3259.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 165 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1134//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1134//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1134//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1134//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1134//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1134//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1134//console
This message is automatically generated.
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: mapreduce-3259.patch
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3259) ContainerLocalizer should get the
proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated MAPREDUCE-3259:
----------------------------------
Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0)
Status: Open (was: Patch Available)
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3259) ContainerLocalizer should get the
proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated MAPREDUCE-3259:
----------------------------------
Fix Version/s: 0.24.0
0.23.0
Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0)
Status: Patch Available (was: Open)
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: mapreduce-3259.patch
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136901#comment-13136901 ]
Hudson commented on MAPREDUCE-3259:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #73 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/73/])
Merge -c 1189667 from trunk to branch-0.23 to fix MAPREDUCE-3259.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189668
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3259) ContainerLocalizer should get the
proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated MAPREDUCE-3259:
----------------------------------
Attachment: mapreduce-3259.patch
The attached patch is for trunk, but should also apply to branch-0.23.
Although the client-side environment may be set by users or by the admin, the framework including what may be needed by the hadoop code base in java.library.path seems more reasonable. After all libhadoop.so may be required to run things even if users don't explicitly use any features that directly depend on it.
Since meaningful tests need specific environment setup and the optional native build, no automated test was added. The risk of this change should be minimal.
Manual testing was carried out as follows.
- Enable JniBasedUnixUserGroupsMapping.
- Build without patch.
- Test failures observed in node manager and job client.
- Apply patch and rebuild.
- Without the library, many tests in node manager and job client failed.
- Installed the library.
- All tests pass.
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Critical
> Attachments: mapreduce-3259.patch
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136642#comment-13136642 ]
Kihwal Lee commented on MAPREDUCE-3259:
---------------------------------------
Sorry, my explanation is rather wrong and confusing. I shouldn't have said app-level. That's totally wrong.
Apps are fine as you said. It's about yarn system level env. When a JVM is directly launched by the LTC, neither of LD_LIBRARY_PATH or java.library.path is set. This is the case for ContainerLocalizer. It will fail to load libhadoop.so if it needs it. If the library is installed in one of the directories ld looks at, it will work without the setting. Or the hadoop lib path may be added to /etc/ld.so.conf or /etc/ld.so.conf.d. Is it still confusing? Or am I totally wrong about this?
I can leave out the second part in the patch for the client since a proper env can be placed in the container script using the facility you mentioned. Only thing I don't like is the default, which assumes the old dir layout. I could try changing the default so that it has a better chance of working right out of the box, but that's not critical.
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3259) ContainerLocalizer should get the
proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated MAPREDUCE-3259:
----------------------------------
Attachment: MAPREDUCE-3259.patch.txt
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136909#comment-13136909 ]
Hudson commented on MAPREDUCE-3259:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1243 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1243/])
MAPREDUCE-3259. Added java.library.path of NodeManager to ContainerLocalizer in LinuxContainerExecutor. Contributed by Kihwal Lee.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189667
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136441#comment-13136441 ]
Kihwal Lee commented on MAPREDUCE-3259:
---------------------------------------
bq. Kihwal, won't it suffice to have the right configs (and defaults) for admin.env?
For executing the task script, that's true. For LTC directly launching JVM to run localizer, either LD_LIBRARY_PATH or -Djava.library.path is need to be set properly. It will appear to work until something requiring the native lib is enabled. When I enabled the JNI-based user groups mapping feature, the localizer wouldn't run.
Back to running task script. The task jvm may have to load libhadoop.so because of a certain cluster-level config. I think we should treat it as an app-level dependency, rather than a run-time dependency of specific client code. Sometimes even admins won't know whether the library will be required by enabling a feature. I thought it will be simpler to add java.library.path to take care of the dependency at the app-level.
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136898#comment-13136898 ]
Hudson commented on MAPREDUCE-3259:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #72 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/72/])
Merge -c 1189667 from trunk to branch-0.23 to fix MAPREDUCE-3259.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189668
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137091#comment-13137091 ]
Hudson commented on MAPREDUCE-3259:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #64 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/64/])
Merge -c 1189667 from trunk to branch-0.23 to fix MAPREDUCE-3259.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189668
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3259) ContainerLocalizer should get the
proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated MAPREDUCE-3259:
----------------------------------
Attachment: mapreduce-3259.patch.txt
Reattaching the same patch with license granted, with the name change for the friendlier mine-type.
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137134#comment-13137134 ]
Hudson commented on MAPREDUCE-3259:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #845 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/845/])
MAPREDUCE-3259. Added java.library.path of NodeManager to ContainerLocalizer in LinuxContainerExecutor. Contributed by Kihwal Lee.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189667
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3259) ContainerLocalizer should get the
proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated MAPREDUCE-3259:
----------------------------------
Component/s: mrv2
Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0)
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136619#comment-13136619 ]
Arun C Murthy commented on MAPREDUCE-3259:
------------------------------------------
Kihwal, sorry if I wasn't clear.
If the admins set mapreduce.admin.user.env such that LD_LIBRARY_PATH includes paths for libhadoop.so and JNI-based user-groups mapping, what goes wrong?
mapreduce.admin.user.env is for cluster-level configs (libhadoop, user-groups mapping etc.) and app-specific deps can go in mapreduce.(map,reduce).env. What am I missing here?
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135015#comment-13135015 ]
Kihwal Lee commented on MAPREDUCE-3259:
---------------------------------------
The javadoc warning is not caused by this patch.
{noformat} [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/JobEndNotifier.java:112: warning - @param argument "config" is not a parameter name.
{noformat}
The findbugs warnings don't seem to be new. Even the ones in the NM and the client app have nothing to do with this patch.
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: mapreduce-3259.patch
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136906#comment-13136906 ]
Hudson commented on MAPREDUCE-3259:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1166 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1166/])
MAPREDUCE-3259. Added java.library.path of NodeManager to ContainerLocalizer in LinuxContainerExecutor. Contributed by Kihwal Lee.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189667
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136330#comment-13136330 ]
Arun C Murthy commented on MAPREDUCE-3259:
------------------------------------------
Kihwal, won't it suffice to have the right configs (and defaults) for admin.env?
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136888#comment-13136888 ]
Hadoop QA commented on MAPREDUCE-3259:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501054/MAPREDUCE-3259.patch.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 170 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryParsing
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1168//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1168//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1168//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1168//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1168//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1168//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1168//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1168//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-web-proxy.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1168//console
This message is automatically generated.
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136870#comment-13136870 ]
Kihwal Lee commented on MAPREDUCE-3259:
---------------------------------------
The new patch contains only the necessary change pertaining to LinuxContainerExecutor#startLocalizer.
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3259) ContainerLocalizer should get
the proper java.library.path from LinuxContainerExecutor
Posted by "Kihwal Lee (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136899#comment-13136899 ]
Kihwal Lee commented on MAPREDUCE-3259:
---------------------------------------
- The test failure is not related to the patch. It fails because the test hard codes the user name.
junit.framework.ComparisonFailure: Incorrect username expected:<[mapred]> but was:<[jenkins]>
- None of the findbugs warnings are related to this patch or new.
- Testing require deployment with a set of specific configurations. As reported above, it was done manually.
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3259
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-3259.patch.txt, mapreduce-3259.patch, mapreduce-3259.patch.txt
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns a JVM for ContainerLocalizer.
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this. This is because tasks' run-time environment is specified in the launch script by client. Setting LD_LIBRARY_PATH is the primary way of specifying the locations of required native library in this case. The config property, mapreduce.admin.user.env is always set in the job environment and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira