You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Wenchen Fan (Jira)" <ji...@apache.org> on 2026/08/07 14:21:00 UTC
[jira] [Resolved] (SPARK-58358) SparkSession.removeTag does not validate tags (Spark Classic)
[ https://issues.apache.org/jira/browse/SPARK-58358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan resolved SPARK-58358.
---------------------------------
Fix Version/s: 4.3.0
Resolution: Fixed
Issue resolved by pull request 57550
[https://github.com/apache/spark/pull/57550]
> SparkSession.removeTag does not validate tags (Spark Classic)
> -------------------------------------------------------------
>
> Key: SPARK-58358
> URL: https://issues.apache.org/jira/browse/SPARK-58358
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Jubin Soni
> Assignee: Jubin Soni
> Priority: Minor
> Labels: pull-request-available
> Fix For: 4.3.0
>
>
> {{SparkSession.removeTag(tag)}} in Spark Classic does not validate its argument, even though:
> * Its API contract documents that tags {*}cannot be empty or contain a comma ({{{}','{}}}){*}.
> * The sibling {{addTag}} API validates input via {{{}SparkContext.throwIfInvalidTag{}}}.
> * The Spark Connect implementation validates both {{addTag}} and {{{}removeTag{}}}.
> * {{SparkContext.removeJobTags}} also validates its input.
> The current Spark Classic implementation removes the tag directly from the managed tag set:
> override def removeTag(tag: String): Unit = managedJobTags.get().remove(tag)
> As a result, calls such as {{removeTag("")}} and {{removeTag("a,b")}} silently succeed instead of throwing {{{}IllegalArgumentException{}}}.
> This is a correctness and consistency issue because:
> * Spark Classic does not enforce its documented API contract.
> * Spark Classic and Spark Connect exhibit different behavior for the same API.
> *Proposed Fix:*
> Update the Spark Classic implementation of `{{{}SparkSession.removeTag{}}}` to call:
> {code:java}
> SparkContext.throwIfInvalidTag(tag){code}
> before removing the tag, mirroring the existing behavior of {{{}addTag{}}}.
>
> *Test Coverage:*
> Add missing validation tests for both implementations:
> * *Scala ({{{}SparkSessionJobTaggingAndCancellationSuite{}}})*
> ** Verify {{addTag}} and {{removeTag}} reject empty and comma-containing tags.
> ** Verify invalid tags are not added to the current tag set.
> ** Verify removing a valid but absent tag remains a no-op.
> * *PySpark ({{{}test_job_cancellation{}}})*
> ** Verify the same API contract in both Spark Classic and Spark Connect (via the shared test mixin).
> ** Additionally, tighten the {{check_job_cancellation}} helper to re-raise non-cancellation exceptions instead of treating any exception as a successful cancellation.
> *Impact:*
> * Test and correctness improvement only.
> * No user-facing API changes beyond enforcing the existing documented contract consistently across Spark Classic and Spark Connect.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org