You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "KWON BYUNGCHANG (JIRA)" <ji...@apache.org> on 2017/06/14 08:34:00 UTC
[jira] [Created] (YARN-6711) log aggregation can't use other
filesystem instead of fs.defaultFS
KWON BYUNGCHANG created YARN-6711:
-------------------------------------
Summary: log aggregation can't use other filesystem instead of fs.defaultFS
Key: YARN-6711
URL: https://issues.apache.org/jira/browse/YARN-6711
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 3.0.0-alpha3, 2.7.1
Reporter: KWON BYUNGCHANG
Priority: Minor
hdfs support federated namespace.
fs.defaultFS should choice one of federated namespace.
however if {{yarn.nodemanager.remote-app-log-dir}} does not belong to default fs,
nodemanager will stop.
absolutely log-aggregation should can use other namespace instead of default fs.
below is error log
configuration is
{quote}
fs.defaultFS=hdfs://x3
yarn.nodemanager.remote-app-log-dir=hdfs://x3v/logs
{quote}
{code}
2017-06-14 15:59:47,792 FATAL org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread
java.lang.IllegalArgumentException: Wrong FS: hdfs://x3v/logs, expected: hdfs://x3
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:646)
at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:194)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:106)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.verifyAndCreateRemoteLogDir(LogAggregationService.java:192)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:319)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:443)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:67)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:175)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:108)
at java.lang.Thread.run(Thread.java:745)
2017-06-14 15:59:47,794 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Exiting, bbye..
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org