[jira] [Updated] (SPARK-13060) CoarsedExecutorBackend register to driver should wait Executor was ready?

["SuYan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-13060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

SuYan updated SPARK-13060:
--------------------------
    Description: 
Hi Josh

I am spark user, currently I feel confused about executor registration.
{code}
Why CoarsedExecutorBackend register to driver first, and after registerDriver successful, then initial Executor.
override def receive: PartialFunction[Any, Unit] = {
  case RegisteredExecutor(hostname) =>
    logInfo("Successfully registered with driver")
    executor = new Executor(executorId, hostname, env, userClassPath, isLocal = false)
​
​{code}

and it is strange that exit(1) while launchTask with executor=null(due to some errors occurs in Executor initialization).
{code}
case LaunchTask(data) =>
  if (executor == null) {
    logError("Received LaunchTask command but executor was null")
    System.exit(1)
{code}
What concerns was considered to make registerDriver first, Why not register to driver until Executor sth. have been all ready?  
like 
{code}

  val (hostname, _) = Utils.extractHostPortFromSparkUrl(hostPort)
  val executor: Executor = new Executor(executorId, hostname, env, userClassPath, isLocal = false)
{code}

  was:
Hi Josh

I am spark user, currently I feel confused about executor registration.
{code}
Why CoarsedExecutorBackend register to driver first, and after registerDriver successful, then initial Executor.
override def receive: PartialFunction[Any, Unit] = {
  case RegisteredExecutor(hostname) =>
    logInfo("Successfully registered with driver")
    executor = new Executor(executorId, hostname, env, userClassPath, isLocal = false)
​
​{code}

and it is strange that exit(1) while launchTask with executor=null(due to some errors occurs in Executor initialization).
{code}
case LaunchTask(data) =>
  if (executor == null) {
    logError("Received LaunchTask command but executor was null")
    System.exit(1)
{code}
What concerns was considered to make registerDriver first, Why not register to driver until Executor sth. have been all ready?  


> CoarsedExecutorBackend register to driver should wait Executor was ready?
> -------------------------------------------------------------------------
>
>                 Key: SPARK-13060
>                 URL: https://issues.apache.org/jira/browse/SPARK-13060
>             Project: Spark
>          Issue Type: Bug
>            Reporter: SuYan
>            Priority: Trivial
>
> Hi Josh
> I am spark user, currently I feel confused about executor registration.
> {code}
> Why CoarsedExecutorBackend register to driver first, and after registerDriver successful, then initial Executor.
> override def receive: PartialFunction[Any, Unit] = {
>   case RegisteredExecutor(hostname) =>
>     logInfo("Successfully registered with driver")
>     executor = new Executor(executorId, hostname, env, userClassPath, isLocal = false)
> ​
> ​{code}
> and it is strange that exit(1) while launchTask with executor=null(due to some errors occurs in Executor initialization).
> {code}
> case LaunchTask(data) =>
>   if (executor == null) {
>     logError("Received LaunchTask command but executor was null")
>     System.exit(1)
> {code}
> What concerns was considered to make registerDriver first, Why not register to driver until Executor sth. have been all ready?  
> like 
> {code}
>   val (hostname, _) = Utils.extractHostPortFromSparkUrl(hostPort)
>   val executor: Executor = new Executor(executorId, hostname, env, userClassPath, isLocal = false)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org