You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by vanzin <gi...@git.apache.org> on 2016/03/04 00:24:08 UTC

[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

GitHub user vanzin opened a pull request:

    https://github.com/apache/spark/pull/11500

    [SPARK-13577] [yarn] Allow Spark jar to be multiple jars, archive.

    In preparation for the demise of assemblies, this change allows the
    YARN backend to use multiple jars and globs as the "Spark jar". The
    config option has been renamed to "spark.yarn.jars" to reflect that.
    
    A second option "spark.yarn.archive" was also added; if set, this
    takes precedence and uploads an archive expected to contain the jar
    files with the Spark code and its dependencies.
    
    Existing deployments should keep working, mostly. This change drops
    support for the "SPARK_JAR" environment variable, and also does not
    fall back to using "jarOfClass" if no configuration is set, falling
    back to finding files under SPARK_HOME instead. This should be fine
    since "jarOfClass" probably wouldn't work unless you were using
    spark-submit anyway.
    
    Tested with the unit tests, and trying the different config options
    on a YARN cluster.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/vanzin/spark SPARK-13577

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/11500.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #11500
    
----
commit da39f4b80c881655f6933002a781ef5db1a0b36a
Author: Marcelo Vanzin <va...@cloudera.com>
Date:   2016-03-03T01:01:29Z

    [SPARK-13577] [yarn] Allow Spark jar to be multiple jars, archive.
    
    In preparation for the demise of assemblies, this change allows the
    YARN backend to use multiple jars and globs as the "Spark jar". The
    config option has been renamed to "spark.yarn.jars" to reflect that.
    
    A second option "spark.yarn.archive" was also added; if set, this
    takes precedence and uploads an archive expected to contain the jar
    files with the Spark code and its dependencies.
    
    Existing deployments should keep working, mostly. This change drops
    support for the "SPARK_JAR" environment variable, and also does not
    fall back to using "jarOfClass" if no configuration is set, falling
    back to finding files under SPARK_HOME instead. This should be fine
    since "jarOfClass" probably wouldn't work unless you were using
    spark-submit anyway.
    
    Tested with the unit tests, and trying the different config options
    on a YARN cluster.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-195153800
  
    **[Test build #52873 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52873/consoleFull)** for PR 11500 at commit [`35e4762`](https://github.com/apache/spark/commit/35e4762f983df3762210b32df9fefe43f33ff2fb).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-194718630
  
    Mostly eyeballing it here, but it looks reasonable to me given its goal. You'll know a lot more about what needs to happen to make this work than I would.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by tgravescs <gi...@git.apache.org>.
Github user tgravescs commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-195371687
  
    +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-193983734
  
    Ping anyone?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by tgravescs <gi...@git.apache.org>.
Github user tgravescs commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11500#discussion_r55734467
  
    --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
    @@ -423,7 +423,63 @@ private[spark] class Client(
         }
     
         /**
    -     * Copy the given main resource to the distributed cache if the scheme is not "local".
    +     * Add Spark to the cache. There are two settings that control what files to add to the cache:
    +     * - if a Spark archive is defined, use the archive. The archive is expected to contain
    +     *   jar files at its root directory.
    +     * - if a list of jars is provided, filter the non-local ones, resolve globs, and
    +     *   add the found files to the cache.
    +     *
    +     * Note that the archive cannot be a "local" URI. If none of the above settings are found,
    +     * then upload all files found in $SPARK_HOME/jars.
    +     *
    +     * TODO: currently the code looks in $SPARK_HOME/lib while the work to replace assemblies
    --- End diff --
    
    do we have jira to explicitly change this when that happens?  can you add comment with that jira.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by tgravescs <gi...@git.apache.org>.
Github user tgravescs commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-195016519
  
    minor nit about comment otherwise lgtm. 
    
    I assume you tested both archives and jars on a real yarn cluster in both cluster/client mode? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-195154003
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52873/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-195115643
  
    **[Test build #52873 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52873/consoleFull)** for PR 11500 at commit [`35e4762`](https://github.com/apache/spark/commit/35e4762f983df3762210b32df9fefe43f33ff2fb).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-192049608
  
    **[Test build #52415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52415/consoleFull)** for PR 11500 at commit [`da39f4b`](https://github.com/apache/spark/commit/da39f4b80c881655f6933002a781ef5db1a0b36a).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-192018298
  
    **[Test build #52415 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52415/consoleFull)** for PR 11500 at commit [`da39f4b`](https://github.com/apache/spark/commit/da39f4b80c881655f6933002a781ef5db1a0b36a).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11500#discussion_r55736339
  
    --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
    @@ -423,7 +423,63 @@ private[spark] class Client(
         }
     
         /**
    -     * Copy the given main resource to the distributed cache if the scheme is not "local".
    +     * Add Spark to the cache. There are two settings that control what files to add to the cache:
    +     * - if a Spark archive is defined, use the archive. The archive is expected to contain
    +     *   jar files at its root directory.
    +     * - if a list of jars is provided, filter the non-local ones, resolve globs, and
    +     *   add the found files to the cache.
    +     *
    +     * Note that the archive cannot be a "local" URI. If none of the above settings are found,
    +     * then upload all files found in $SPARK_HOME/jars.
    +     *
    +     * TODO: currently the code looks in $SPARK_HOME/lib while the work to replace assemblies
    --- End diff --
    
    That's SPARK-13579. I'll have to fix all these when working on that one, otherwise Spark won't work, so these are mostly reminders for myself that will go away pretty soon.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-193490730
  
    **[Test build #52607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52607/consoleFull)** for PR 11500 at commit [`9bab2ea`](https://github.com/apache/spark/commit/9bab2ea1fc5bbb91497e1994b3613cd6cdc4b3be).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/11500


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-192049786
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52415/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-195153999
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11500#discussion_r55726981
  
    --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
    @@ -653,7 +653,9 @@ private[spark] object SparkConf extends Logging {
         "spark.memory.offHeap.enabled" -> Seq(
           AlternateConfig("spark.unsafe.offHeap", "1.6")),
         "spark.rpc.message.maxSize" -> Seq(
    -      AlternateConfig("spark.akka.frameSize", "1.6"))
    +      AlternateConfig("spark.akka.frameSize", "1.6")),
    +    "spark.yarn.jars" -> Seq(
    +      AlternateConfig("spark.yarn.jar", "2.0"))
    --- End diff --
    
    No, `.archive` is a new functionality that @tgravescs asked for during review of the spec; it behaves differently from the other two. `.jars` has the same functionality as the old `.jar` and then some more.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-193542877
  
    **[Test build #52607 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52607/consoleFull)** for PR 11500 at commit [`9bab2ea`](https://github.com/apache/spark/commit/9bab2ea1fc5bbb91497e1994b3613cd6cdc4b3be).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-195017181
  
    > both archives and jars on a real yarn cluster in both cluster/client mode
    
    Not sure I tested client mode, but I'll take a look. Shouldn't really make a difference, but you never know.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-192049783
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-193543521
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-193543523
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52607/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the pull request:

    https://github.com/apache/spark/pull/11500#issuecomment-195115326
  
    All good in all modes, just find a broken log message.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13577] [yarn] Allow Spark jar to be mul...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11500#discussion_r55643032
  
    --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
    @@ -653,7 +653,9 @@ private[spark] object SparkConf extends Logging {
         "spark.memory.offHeap.enabled" -> Seq(
           AlternateConfig("spark.unsafe.offHeap", "1.6")),
         "spark.rpc.message.maxSize" -> Seq(
    -      AlternateConfig("spark.akka.frameSize", "1.6"))
    +      AlternateConfig("spark.akka.frameSize", "1.6")),
    +    "spark.yarn.jars" -> Seq(
    +      AlternateConfig("spark.yarn.jar", "2.0"))
    --- End diff --
    
    I'm probably wrong about this, but is it `.archive` that is the new name for `.jar` rather than `.jars`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org