You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by tnachen <gi...@git.apache.org> on 2014/07/29 01:15:21 UTC

[GitHub] spark pull request: [SPARK-2022] Fix extra java config to spark cl...

GitHub user tnachen opened a pull request:

    https://github.com/apache/spark/pull/1622

    [SPARK-2022] Fix extra java config to spark class command via mesos

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tnachen/spark fix_mesos_extra_config

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/1622.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1622
    
----
commit eae354f2c5889b831be73c3fe357f1d6fd618a30
Author: Timothy Chen <tn...@gmail.com>
Date:   2014-07-22T07:07:35Z

    Fix extra java config to spark class command via mesos

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] spark pull request: [SPARK-2022] Fix extra java config to spark cl...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/1622#issuecomment-50419093
  
    QA results for PR 1622:<br>- This patch PASSES unit tests.<br>- This patch merges cleanly<br>- This patch adds no public classes<br><br>For more information see test ouptut:<br>https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17314/consoleFull


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] spark pull request: [SPARK-2022] Fix extra java config to spark cl...

Posted by pwendell <gi...@git.apache.org>.

Github user pwendell commented on a diff in the pull request:

    https://github.com/apache/spark/pull/1622#discussion_r15662427
  
    --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala ---
    @@ -139,15 +139,16 @@ private[spark] class CoarseMesosSchedulerBackend(
         if (uri == null) {
           val runScript = new File(sparkHome, "./bin/spark-class").getCanonicalPath
           command.setValue(
    -        "\"%s\" org.apache.spark.executor.CoarseGrainedExecutorBackend %s %s %s %s %d".format(
    -          runScript, extraOpts, driverUrl, offer.getSlaveId.getValue, offer.getHostname, numCores))
    +        ("SPARK_JAVA_OPTS='%s' \"%s\" " +
    --- End diff --
    
    If you look at the internals of `spark-class`, I think you want to be setting `SPARK_EXECUTOR_OPTS` here. Also, I think environment variables should be set via `setEnvironment` as shown above, not like this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---