You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Thomas Graves (JIRA)" <ji...@apache.org> on 2016/04/08 16:51:25 UTC
[jira] [Updated] (SPARK-14279) Improve the spark build to pick the
version information from the pom file instead of package.scala
[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves updated SPARK-14279:
----------------------------------
Description:
Right now the spark-submit --version and other parts of the code pick up version information from a static SPARK_VERSION. We would want to pick the version from the pom.version probably stored inside a properties file. Also, it might be nice to have other details like branch, build information and other specific details when having a spark-submit --version
the motivation is to more easily tie this to automated continuous integration and deployment and to easily have traceability.
Part of this is right now you have to manually change a java file to change the version that comes out when you run spark-submit --version. With continuous integration the build numbers could be something like 1.6.1.X (where X increments on each change) and I want to see the exact version easily. Having to manually change a java file makes that hard. obviously that should make the apache spark releases easier as you don't have to manually change this file as well.
The other important part for me is the git information. This easily lets me trace it back to exact commits. We have a multi-tenant YARN cluster and users can run many different versions at once. I want to be able to see exactly which version they are running. The reason to know exact version can range from helping debug some problem to making sure someone didn't hack something in Spark to cause bad things (generally they should use approved version), etc.
was:Right now the spark-submit --version and other parts of the code pick up version information from a static SPARK_VERSION. We would want to pick the version from the pom.version probably stored inside a properties file. Also, it might be nice to have other details like branch, build information and other specific details when having a spark-submit --version
> Improve the spark build to pick the version information from the pom file instead of package.scala
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-14279
> URL: https://issues.apache.org/jira/browse/SPARK-14279
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Reporter: Sanket Reddy
> Assignee: Sanket Reddy
> Priority: Minor
>
> Right now the spark-submit --version and other parts of the code pick up version information from a static SPARK_VERSION. We would want to pick the version from the pom.version probably stored inside a properties file. Also, it might be nice to have other details like branch, build information and other specific details when having a spark-submit --version
> the motivation is to more easily tie this to automated continuous integration and deployment and to easily have traceability.
> Part of this is right now you have to manually change a java file to change the version that comes out when you run spark-submit --version. With continuous integration the build numbers could be something like 1.6.1.X (where X increments on each change) and I want to see the exact version easily. Having to manually change a java file makes that hard. obviously that should make the apache spark releases easier as you don't have to manually change this file as well.
> The other important part for me is the git information. This easily lets me trace it back to exact commits. We have a multi-tenant YARN cluster and users can run many different versions at once. I want to be able to see exactly which version they are running. The reason to know exact version can range from helping debug some problem to making sure someone didn't hack something in Spark to cause bad things (generally they should use approved version), etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org