You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Wenchen Fan (Jira)" <ji...@apache.org> on 2026/08/07 14:21:00 UTC

[jira] [Resolved] (SPARK-58358) SparkSession.removeTag does not validate tags (Spark Classic)

     [ https://issues.apache.org/jira/browse/SPARK-58358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenchen Fan resolved SPARK-58358.
---------------------------------
    Fix Version/s: 4.3.0
       Resolution: Fixed

Issue resolved by pull request 57550
[https://github.com/apache/spark/pull/57550]

> SparkSession.removeTag does not validate tags (Spark Classic)
> -------------------------------------------------------------
>
>                 Key: SPARK-58358
>                 URL: https://issues.apache.org/jira/browse/SPARK-58358
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 4.0.0
>            Reporter: Jubin Soni
>            Assignee: Jubin Soni
>            Priority: Minor
>              Labels: pull-request-available
>             Fix For: 4.3.0
>
>
> {{SparkSession.removeTag(tag)}} in Spark Classic does not validate its argument, even though:
>  * Its API contract documents that tags {*}cannot be empty or contain a comma ({{{}','{}}}){*}.
>  * The sibling {{addTag}} API validates input via {{{}SparkContext.throwIfInvalidTag{}}}.
>  * The Spark Connect implementation validates both {{addTag}} and {{{}removeTag{}}}.
>  * {{SparkContext.removeJobTags}} also validates its input.
> The current Spark Classic implementation removes the tag directly from the managed tag set:
> override def removeTag(tag: String): Unit = managedJobTags.get().remove(tag)
> As a result, calls such as {{removeTag("")}} and {{removeTag("a,b")}} silently succeed instead of throwing {{{}IllegalArgumentException{}}}.
> This is a correctness and consistency issue because:
>  * Spark Classic does not enforce its documented API contract.
>  * Spark Classic and Spark Connect exhibit different behavior for the same API.
> *Proposed Fix:*
> Update the Spark Classic implementation of `{{{}SparkSession.removeTag{}}}` to call:
> {code:java}
> SparkContext.throwIfInvalidTag(tag){code}
> before removing the tag, mirroring the existing behavior of {{{}addTag{}}}.
>  
> *Test Coverage:*
> Add missing validation tests for both implementations:
>  * *Scala ({{{}SparkSessionJobTaggingAndCancellationSuite{}}})*
>  ** Verify {{addTag}} and {{removeTag}} reject empty and comma-containing tags.
>  ** Verify invalid tags are not added to the current tag set.
>  ** Verify removing a valid but absent tag remains a no-op.
>  * *PySpark ({{{}test_job_cancellation{}}})*
>  ** Verify the same API contract in both Spark Classic and Spark Connect (via the shared test mixin).
>  ** Additionally, tighten the {{check_job_cancellation}} helper to re-raise non-cancellation exceptions instead of treating any exception as a successful cancellation.
> *Impact:*
>  * Test and correctness improvement only.
>  * No user-facing API changes beyond enforcing the existing documented contract consistently across Spark Classic and Spark Connect.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org