[jira] [Created] (HDFS-15465) Support WebHDFS accesses to the data stored in secure Datanode through insecure Namenode

["Toshihiko Uchida (Jira)" <ji...@apache.org>]

Toshihiko Uchida created HDFS-15465:
---------------------------------------

             Summary: Support WebHDFS accesses to the data stored in secure Datanode through insecure Namenode
                 Key: HDFS-15465
                 URL: https://issues.apache.org/jira/browse/HDFS-15465
             Project: Hadoop HDFS
          Issue Type: Wish
          Components: federation, webhdfs
            Reporter: Toshihiko Uchida
            Assignee: Toshihiko Uchida
         Attachments: webhdfs-federation.pdf

We're federating a secure HDFS cluster with an insecure cluster.
Using HDFS RPC, we can access the data managed by insecure Namenode and stored in secure Datanode.
However, it does not work for WebHDFS due to HadoopIllegalArgumentException.
{code}
$ curl -i "http://<INSECURE_NAMENODE>:<PORT>/webhdfs/v1/<PATH>?op=OPEN"
HTTP/1.1 307 TEMPORARY_REDIRECT
(omitted)
Location: http://<SECURE_DATANODE>:<PORT>/webhdfs/v1/<PATH>?op=OPEN&namenoderpcaddress=<INSECURE_NAMENODE>&offset=0
$ curl -i "http://<SECURE_DATANODE>:<PORT>/webhdfs/v1/<PATH>?op=OPEN&namenoderpcaddress=<INSECURE_NAMENODE>&offset=0"
HTTP/1.1 400 Bad Request
(omitted)
{"RemoteException":{"exception":"HadoopIllegalArgumentException","javaClassName":"org.apache.hadoop.HadoopIllegalArgumentException","message":"Invalid argument, newValue is null"}}
{code}
This is because secure Datanode expects a delegation token, but insecure Namenode does not return it to a client.
- org.apache.hadoop.security.token.Token.decodeWritable
{code}
  private static void decodeWritable(Writable obj,
                                     String newValue) throws IOException {
    if (newValue == null) {
      throw new HadoopIllegalArgumentException(
              "Invalid argument, newValue is null");
    }
{code}

The issue proposes to support the access also for WebHDFS.
The attached PDF file [^webhdfs-federation.pdf] depicts our current architecture and proposal.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org