You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "BoYang (JIRA)" <ji...@apache.org> on 2014/05/14 02:55:18 UTC
[jira] [Created] (HADOOP-10606) NodeManager cannot launch container
when using RawLocalFileSystem for fs.file.impl
BoYang created HADOOP-10606:
-------------------------------
Summary: NodeManager cannot launch container when using RawLocalFileSystem for fs.file.impl
Key: HADOOP-10606
URL: https://issues.apache.org/jira/browse/HADOOP-10606
Project: Hadoop Common
Issue Type: Bug
Components: fs, io, util
Affects Versions: 2.4.0
Environment: The environment does not matter with this issue. But I use Windows 8 64bits, Open JDK 7.0.
Reporter: BoYang
Priority: Critical
Fix For: 2.4.0
Node manager failed to launch container when I set fs.file.impl to org.apache.hadoop.fs.RawLocalFileSystem in core-site.xml.
The log is:
WARN ContainersLauncher #11 org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch - Failed to launch container.
java.lang.ClassCastException: org.apache.hadoop.fs.RawLocalFileSystem cannot be cast to org.apache.hadoop.fs.LocalFileSystem
at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:339)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:270)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:344)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLogPathForWrite(LocalDirsHandlerService.java:307)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:185)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
The issue is in hadoop-common-project\hadoop-common\src\main\java\org\apache\hadoop\fs\LocalDirAllocator.java. It invokes FileSystem.getLocal(), which tries to convert the FileSystem to LocalFileSystem and will fail when FileSystem object is RawLocalFileSystem (RawLocalFileSystem is not a sub class of LocalFileSystem).
public static LocalFileSystem getLocal(Configuration conf)
throws IOException {
return (LocalFileSystem)get(LocalFileSystem.NAME, conf);
}
The fix for LocalDirAllocator.java seems to be invoking LocalFileSystem.get() instead of LocalFileSystem.getLocal()?
--
This message was sent by Atlassian JIRA
(v6.2#6252)