[jira] [Created] (HDFS-16577) Let administrator override connection details when registering datanodes

["Teo Klestrup Röijezon (Jira)" <ji...@apache.org>]

Teo Klestrup Röijezon created HDFS-16577:
--------------------------------------------

             Summary: Let administrator override connection details when registering datanodes
                 Key: HDFS-16577
                 URL: https://issues.apache.org/jira/browse/HDFS-16577
             Project: Hadoop HDFS
          Issue Type: New Feature
          Components: datanode, namenode
    Affects Versions: 3.2.2
            Reporter: Teo Klestrup Röijezon


Currently (as of 3.2.2, but reading through the release notes this doesn't seem to have changed since then) DataNodes use the same properties for deciding which port to bind each service to, as for deciding which ports are included in the `DatanodeRegistration` sent to the NameNode. Further, NameNodes overwrite the DataNode's IP address with the incoming address during registration.

Both of these prevent external users from connecting to DataNodes that are hosted behind some sort of NAT (such as Kubernetes).

I have created a spike branch ([https://github.com/stackabletech/hadoop/tree/spike/override-datanode-id,] based on v3.2.2) that I have confirmed solves this problem for us. There's clearly some work to be done integrating this properly (such as using the regular Hadoop config system and falling back to the old behaviour if no override is configured). I'd be happy to take that on to the best of my ability (with the caveats that I'm not super familiar with the Hadoop codebase, and that my Java is quite rusty at this point) if the overall direction seems acceptable.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org