You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Stefaan Lippens (JIRA)" <ji...@apache.org> on 2019/03/06 15:05:00 UTC
[jira] [Created] (HDFS-14341) Weird handling of plus sign in paths
in WebHDFS REST API
Stefaan Lippens created HDFS-14341:
--------------------------------------
Summary: Weird handling of plus sign in paths in WebHDFS REST API
Key: HDFS-14341
URL: https://issues.apache.org/jira/browse/HDFS-14341
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 3.1.1
Reporter: Stefaan Lippens
We're using Hadoop 3.1.1 at the moment and have an issue with the handling of paths that contain plus signs (generated by Kafka HDFS Connector).
For example, I created this example directory {{tmp/plus+plus}}
{code:java}
$ hadoop fs -ls tmp/plus+plus
Found 1 items
-rw-r--r-- 3 stefaan supergroup 7079 2019-03-06 14:31 tmp/plus+plus/foo.txt{code}
When trying to list this folder through WebHDFS the naive way:
{code:java}
$ curl 'http://hadoopname05:9870/webhdfs/v1/user/stefaan/tmp/plus+plus?user.name=stefaan&op=LISTSTATUS'
{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File /user/stefaan/tmp/plus plus does not exist."}}{code}
Fair enough, the plus sign {{+}} is a special character in URLs, let's encode it as {{%2B}}:
{code:java}
$ curl 'http://hadoopname05:9870/webhdfs/v1/user/stefaan/tmp/plus%2Bplus?user.name=stefaan&op=LISTSTATUS'
{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File /user/stefaan/tmp/plus plus does not exist."}}{code}
Doesn't work.
After some trial and error I found that I could get it working by encode the thing twice ({{"+" -> "%2B" -> "%252B"}}):
{code:java}
curl 'http://hadoopname05:9870/webhdfs/v1/user/stefaan/tmp/plus%252Bplus?user.name=stefaan&op=LISTSTATUS'
{"FileStatuses":{"FileStatus":[
{"accessTime":1551882704527,"blockSize":134217728,"childrenNum":0,"fileId":314914,"group":"supergroup","length":7079,"modificationTime":1551882704655,"owner":"stefaan","pathSuffix":"foo.txt","permission":"644","replication":3,"storagePolicy":0,"type":"FILE"}
]}}{code}
Seems like there is some double decoding going on in WebHDFS REST API.
I also tried with some other special characters like {{@}} and {{=}}, and for these it seems to work both when encoding once ({{%40}} and {{%3D}} respectively) and encoding twice ({{%2540}} and {{%253D}} respectively)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org