You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "GANHONGNAN (Jira)" <ji...@apache.org> on 2026/07/27 03:41:00 UTC

[jira] [Updated] (SPARK-58362) [CORE] bug report: collision of resource profile id

     [ https://issues.apache.org/jira/browse/SPARK-58362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

GANHONGNAN updated SPARK-58362:
-------------------------------
    Description: 
Spark's default resource profile id is 0. In stage level resource profile, the first returned resource id is also 0 which would lead to a collision.
val UNKNOWN_RESOURCE_PROFILE_ID = -1
val DEFAULT_RESOURCE_PROFILE_ID = 0

private lazy val nextProfileId = new AtomicInteger(0) \{{ }}
 
[spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L365]

Line 365
|private lazy val nextProfileId = new AtomicInteger(0)|

 

 
[spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L363]

Line 363
|val DEFAULT_RESOURCE_PROFILE_ID = 0|

 
 
 

  was:
Spark's default resource profile id is 0. In stage level resource profile, the first returned resource id is also 0 which would lead to a collision.
val UNKNOWN_RESOURCE_PROFILE_ID = -1
val DEFAULT_RESOURCE_PROFILE_ID = 0

private lazy val nextProfileId = new AtomicInteger(0) {{  }}
 
[spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L365]

Line 365 in [f883693|https://github.com/apache/spark/commit/f8836939c6a89bd788ad648ec30df757cde204ac]
|private lazy val nextProfileId = new AtomicInteger(0)|

 

 
[spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L363]

Line 363 in [f883693|https://github.com/apache/spark/commit/f8836939c6a89bd788ad648ec30df757cde204ac]
|val DEFAULT_RESOURCE_PROFILE_ID = 0|

 
 
 


> [CORE] bug report: collision of resource profile id
> ---------------------------------------------------
>
>                 Key: SPARK-58362
>                 URL: https://issues.apache.org/jira/browse/SPARK-58362
>             Project: Spark
>          Issue Type: Bug
>          Components: Bug
>    Affects Versions: 3.2.1
>            Reporter: GANHONGNAN
>            Priority: Critical
>
> Spark's default resource profile id is 0. In stage level resource profile, the first returned resource id is also 0 which would lead to a collision.
> val UNKNOWN_RESOURCE_PROFILE_ID = -1
> val DEFAULT_RESOURCE_PROFILE_ID = 0
> private lazy val nextProfileId = new AtomicInteger(0) \{{ }}
>  
> [spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L365]
> Line 365
> |private lazy val nextProfileId = new AtomicInteger(0)|
>  
>  
> [spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L363]
> Line 363
> |val DEFAULT_RESOURCE_PROFILE_ID = 0|
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org