You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Marcelo Vanzin (JIRA)" <ji...@apache.org> on 2019/07/16 17:57:00 UTC
[jira] [Resolved] (SPARK-27959) Change YARN resource configs to use
.amount
[ https://issues.apache.org/jira/browse/SPARK-27959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Vanzin resolved SPARK-27959.
------------------------------------
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24989
[https://github.com/apache/spark/pull/24989]
> Change YARN resource configs to use .amount
> -------------------------------------------
>
> Key: SPARK-27959
> URL: https://issues.apache.org/jira/browse/SPARK-27959
> Project: Spark
> Issue Type: Story
> Components: YARN
> Affects Versions: 3.0.0
> Reporter: Thomas Graves
> Assignee: Thomas Graves
> Priority: Major
> Fix For: 3.0.0
>
>
> we are adding in generic resource support into spark where we have suffix for the amount of the resource so that we could support other configs.
> Spark on yarn already had added configs to request resources via the configs spark.yarn.\{executor/driver/am}.resource=<some amount>, where the <some amont> is value and unit together. We should change those configs to have a .amount suffix on them to match the spark configs and to allow future configs to be more easily added. YARN itself already supports tags and attributes so if we want the user to be able to pass those from spark at some point having a suffix makes sense.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org