You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Mingyu Kim (JIRA)" <ji...@apache.org> on 2016/06/15 21:23:09 UTC

[jira] [Created] (SPARK-15974) Create a socket on YARN AM start-up

Mingyu Kim created SPARK-15974:
----------------------------------

             Summary: Create a socket on YARN AM start-up
                 Key: SPARK-15974
                 URL: https://issues.apache.org/jira/browse/SPARK-15974
             Project: Spark
          Issue Type: New Feature
          Components: YARN
            Reporter: Mingyu Kim


YARN provides a way for AppilcationMaster to register a RPC port so that a client outside the YARN cluster can reach the application for any RPCs, but Spark’s YARN AMs simply register a dummy port number of 0. For the Spark programs that starts up a server, this makes it hard for the submitter to discover the server port securely. Spark's ApplicationMaster should optionally create a ServerSocket and pass it to the Spark user program. This socket initialization should be disabled by default.

Some discussion on dev@spark thread: http://apache-spark-developers-list.1001551.n3.nabble.com/Utilizing-YARN-AM-RPC-port-field-td17892.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org