[jira] [Commented] (SPARK-3445) Deprecate and later remove YARN alpha support

["Chester (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168491#comment-14168491 ] 

Chester commented on SPARK-3445:
--------------------------------

What does  Deprecate YARN-alpha  implies ? 

1) No longer active develop and maintained for sure. 
2) When we submit PR, should we make sure they compile for Yarn-alpha ? 

I am writing a PR that requires Spark Yarn Client API changes. should I worries that yarn-alpha Client API changes as well ? 

Currently in master branch, the yarn-alpha doesn't even compile, for example 
sbt/sbt -Pyarn-alpha -Dhadoop.version=2.0.6-alpha

[error] /Users/chester/projects/alpine/spark/yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:125: type mismatch;
[error]  found   : Object
[error]  required: String
[error]     Option(report.getClientToken).getOrElse("")
[error]                                            ^
[error] one error found
[error] (streaming-flume-sink/compile:compile) Compilation failed
[error] (yarn-alpha/compile:compile) Compilation failed
[error] Total time: 9 s, completed Oct 11, 2014 7:57:30 PM


This is certainly a bug as the stable version would be 
 Option(report.getClientToken).map(_toString).getOrElse("")

Should this be fixed ? 

Also, I got  the following while doing

sbt/sbt -Pyarn-alpha -Dhadoop.version=2.0.6-alpha
 test:compile

...
[error] 6 errors found
[error] (streaming-flume-sink/compile:compile) Compilation failed
[error] Total time: 3 s, completed Oct 11, 2014 8:02:57 PM

All the errors are pom.xml dependency related. I don't get the same error when I do compile yarn-stable version.

Advice ? 
 


> Deprecate and later remove YARN alpha support
> ---------------------------------------------
>
>                 Key: SPARK-3445
>                 URL: https://issues.apache.org/jira/browse/SPARK-3445
>             Project: Spark
>          Issue Type: Improvement
>          Components: YARN
>            Reporter: Patrick Wendell
>
> This will depend a bit on both user demand and the commitment level of maintainers, but I'd like to propose the following timeline for yarn-alpha support.
> Spark 1.2: Deprecate YARN-alpha
> Spark 1.3: Remove YARN-alpha (i.e. require YARN-stable)
> Since YARN-alpha is clearly identified as an alpha API, it seems reasonable to drop support for it in a minor release. However, it does depend a bit whether anyone uses this outside of Yahoo!, and that I'm not sure of. In the past this API has been used and maintained by Yahoo, but they'll be migrating soon to the stable API's.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org