You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Chris Nauroth (JIRA)" <ji...@apache.org> on 2013/08/08 19:39:49 UTC
[jira] [Updated] (YARN-1025) ResourceManager and NodeManager do not
load native libraries on Windows.
[ https://issues.apache.org/jira/browse/YARN-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Nauroth updated YARN-1025:
--------------------------------
Component/s: resourcemanager
Description: ResourceManager and NodeManager do not have the correct setting for java.library.path when launched on Windows. This prevents the processes from loading native code from hadoop.dll. The native code is required for correct functioning on Windows (not optional), so this ultimately can cause failures. (was: Neither the NodeManager process itself nor the child container processes that it launches have the correct setting for java.library.path on Windows. This prevents the processes from loading native code from hadoop.dll. The native code is required for correct functioning on Windows (not optional), so this ultimately can cause failures in MapReduce jobs.)
Summary: ResourceManager and NodeManager do not load native libraries on Windows. (was: NodeManager does not propagate java.library.path to launched child containers on Windows)
Kihwal and Chuan, thanks for the comments here and in MAPREDUCE-5451. I think the conclusion then is:
# YARN applications, including MR, are responsible for managing the library path of their launched containers via configuration of LD_LIBRARY_PATH on Linux or PATH on Windows.
# However, there is still a need for the YARN daemons themselves to launch with a correct library path for finding libhadoop.so or hadoop.dll. The other Hadoop daemons automatically find the library and set java.library.path, so ResourceManager and NodeManager need to do the same. They already do this on Linux in the yarn shell script, so we just need to update the cmd scripts with equivalent logic.
I'm leaving this issue open for just the cmd script changes, but we won't make changes for NodeManager to propagate java.library.path to its launched child containers.
I've updated the issue summary and description accordingly.
> ResourceManager and NodeManager do not load native libraries on Windows.
> ------------------------------------------------------------------------
>
> Key: YARN-1025
> URL: https://issues.apache.org/jira/browse/YARN-1025
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager, resourcemanager
> Affects Versions: 3.0.0, 2.1.1-beta
> Reporter: Chris Nauroth
>
> ResourceManager and NodeManager do not have the correct setting for java.library.path when launched on Windows. This prevents the processes from loading native code from hadoop.dll. The native code is required for correct functioning on Windows (not optional), so this ultimately can cause failures.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira