You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Andrew Or (JIRA)" <ji...@apache.org> on 2014/09/12 23:46:36 UTC
[jira] [Commented] (SPARK-3107) Don't pass null jar to executor in
yarn-client mode
[ https://issues.apache.org/jira/browse/SPARK-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132164#comment-14132164 ]
Andrew Or commented on SPARK-3107:
----------------------------------
I see. Yes, setting it to an empty value is semantically different from not setting it, and that is exactly the reason why we shouldn't set these to empty strings when what we really mean to do is to not set it. I think we still do that in a few places here and there.
> Don't pass null jar to executor in yarn-client mode
> ---------------------------------------------------
>
> Key: SPARK-3107
> URL: https://issues.apache.org/jira/browse/SPARK-3107
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.1.0
> Reporter: Andrew Or
>
> In the following line, ExecutorLauncher's `--jar` takes in null.
> {code}
> 14/08/18 20:52:43 INFO yarn.Client: command: $JAVA_HOME/bin/java -server -Xmx512m ... org.apache.spark.deploy.yarn.ExecutorLauncher --class 'notused' --jar null --arg 'ip-172-31-0-12.us-west-2.compute.internal:56838' --executor-memory 1024 --executor-cores 1 --num-executors 2
> {code}
> Also it appears that we set a bunch of system properties to empty strings (not shown). We should avoid setting these if they don't actually contain values.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org