You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Aoyuan Liao (Jira)" <ji...@apache.org> on 2020/10/22 00:04:00 UTC
[jira] [Commented] (SPARK-19297) Add ability for --packages tag to
pull latest version
[ https://issues.apache.org/jira/browse/SPARK-19297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218660#comment-17218660 ]
Aoyuan Liao commented on SPARK-19297:
-------------------------------------
latest.release can be used.
{code:java}
//
bin/spark-submit --packages com.databricks:spark-csv_2.11:latest.release examples/src/main/python/pi.py 10bin/spark-submit --packages com.databricks:spark-csv_2.11:latest.release examples/src/main/python/pi.py 10:: loading settings :: url = jar:file:/home/eve/repo/spark/assembly/target/scala-2.12/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xmlIvy Default Cache set to: /home/eve/.ivy2/cacheThe jars for the packages stored in: /home/eve/.ivy2/jarscom.databricks#spark-csv_2.11 added as a dependency:: resolving dependencies :: org.apache.spark#spark-submit-parent-a24e5fe4-814e-48d0-baf4-6ff489520dd4;1.0 confs: [default] found com.databricks#spark-csv_2.11;1.5.0 in central [1.5.0] com.databricks#spark-csv_2.11;latest.release found org.apache.commons#commons-csv;1.1 in central found com.univocity#univocity-parsers;1.5.1 in centraldownloading https://repo1.maven.org/maven2/com/databricks/spark-csv_2.11/1.5.0/spark-csv_2.11-1.5.0.jar ... [SUCCESSFUL ] com.databricks#spark-csv_2.11;1.5.0!spark-csv_2.11.jar (87ms)downloading https://repo1.maven.org/maven2/org/apache/commons/commons-csv/1.1/commons-csv-1.1.jar ... [SUCCESSFUL ] org.apache.commons#commons-csv;1.1!commons-csv.jar (36ms)downloading https://repo1.maven.org/maven2/com/univocity/univocity-parsers/1.5.1/univocity-parsers-1.5.1.jar ... [SUCCESSFUL ] com.univocity#univocity-parsers;1.5.1!univocity-parsers.jar (127ms):: resolution report :: resolve 1729ms :: artifacts dl 257ms :: modules in use: com.databricks#spark-csv_2.11;1.5.0 from central in [default] com.univocity#univocity-parsers;1.5.1 from central in [default] org.apache.commons#commons-csv;1.1 from central in [default] --------------------------------------------------------------------- | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| --------------------------------------------------------------------- | default | 3 | 3 | 3 | 0 || 3 | 3 | ---------------------------------------------------------------------
{code}
> Add ability for --packages tag to pull latest version
> -----------------------------------------------------
>
> Key: SPARK-19297
> URL: https://issues.apache.org/jira/browse/SPARK-19297
> Project: Spark
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: Steven Landes
> Priority: Minor
> Labels: features, newbie
> Attachments: packages_latest.txt
>
>
> It would be super-convenient, in a development environment, to be able to use the --packages argument to point spark to the latest version of a package instead of specifying a specific version.
> For example, instead of the following:
> --packages com.databricks:spark-csv_2.11:1.5.0
> I could just put in this:
> --packages com.databricks:spark-csv_2.11:latest
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org