You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Ctest (Jira)" <ji...@apache.org> on 2020/04/23 23:36:00 UTC
[jira] [Comment Edited] (HADOOP-17011) Tolerate leading and
trailing spaces in fs.defaultFS
[ https://issues.apache.org/jira/browse/HADOOP-17011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091020#comment-17091020 ]
Ctest edited comment on HADOOP-17011 at 4/23/20, 11:35 PM:
-----------------------------------------------------------
[~liuml07]
I searched around in the hadoop-trunk and found one more place of `conf.get(FS_DEFAULT_NAME_KEY)`. I can change that one together in a new patch.
And interestingly, in ServiceScheduler.java, the code is using `getTrimmed(FS_DEFAULT_NAME_KEY)`.
was (Author: ctest.team):
[~liuml07]
I searched around in the hadoop-trunk and found one more place of `conf.get(FS_DEFAULT_NAME_KEY)`. I can change that one together in a new patch.
Also, in ServiceScheduler.java, the code is using `getTrimmed(FS_DEFAULT_NAME_KEY)`.
> Tolerate leading and trailing spaces in fs.defaultFS
> ----------------------------------------------------
>
> Key: HADOOP-17011
> URL: https://issues.apache.org/jira/browse/HADOOP-17011
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
> Reporter: Ctest
> Assignee: Ctest
> Priority: Major
> Attachments: HADOOP-17011-001.patch
>
>
> *Problem:*
> Currently, `getDefaultUri` is using `conf.get` to get the value of `fs.defaultFS`, which means that the trailing whitespace after a valid URI won’t be removed and could stop namenode and datanode from starting up.
>
> *How to reproduce (Hadoop-2.8.5):*
> Set the configuration
> {code:java}
> <property>
> <name>fs.defaultFS</name>
> <value>hdfs://localhost:9000 </value>
> </property>{code}
> In core-site.xml (there is a whitespace after 9000) and start HDFS.
> Namenode and datanode won’t start and the log message is:
> {code:java}
> 2020-04-23 11:09:48,198 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
> java.lang.IllegalArgumentException: Illegal character in authority at index 7: hdfs://localhost:9000
> at java.net.URI.create(URI.java:852)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.setClientNamenodeAddress(NameNode.java:440)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:897)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:885)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1626)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1694)
> Caused by: java.net.URISyntaxException: Illegal character in authority at index 7: hdfs://localhost:9000
> at java.net.URI$Parser.fail(URI.java:2848)
> at java.net.URI$Parser.parseAuthority(URI.java:3186)
> at java.net.URI$Parser.parseHierarchical(URI.java:3097)
> at java.net.URI$Parser.parse(URI.java:3053)
> at java.net.URI.<init>(URI.java:588)
> at java.net.URI.create(URI.java:850)
> ... 5 more
> {code}
>
> *Solution:*
> Use `getTrimmed` instead of `get` for `fs.defaultFS`:
> {code:java}
> public static URI getDefaultUri(Configuration conf) {
> URI uri =
> URI.create(fixName(conf.getTrimmed(FS_DEFAULT_NAME_KEY, DEFAULT_FS)));
> if (uri.getScheme() == null) {
> throw new IllegalArgumentException("No scheme in default FS: " + uri);
> }
> return uri;
> }
> {code}
> I have submitted a patch for trunk about this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org