You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Wenchen Fan (Jira)" <ji...@apache.org> on 2025/09/01 03:34:00 UTC

[jira] [Assigned] (SPARK-53329) Process all sent-over artifacts rather than failing early for modification attempts

     [ https://issues.apache.org/jira/browse/SPARK-53329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenchen Fan reassigned SPARK-53329:
-----------------------------------

    Assignee: Venkata Sai Akhil Gudesa

> Process all sent-over artifacts rather than failing early for modification attempts
> -----------------------------------------------------------------------------------
>
>                 Key: SPARK-53329
>                 URL: https://issues.apache.org/jira/browse/SPARK-53329
>             Project: Spark
>          Issue Type: Improvement
>          Components: Connect
>    Affects Versions: 4.1.0, 4.0.1
>            Reporter: Venkata Sai Akhil Gudesa
>            Assignee: Venkata Sai Akhil Gudesa
>            Priority: Major
>              Labels: pull-request-available
>
> When a user sends multiple artifacts with the `addArtifacts` API, we process each artifact one at a time on the server-side.
> If the server detects the user attempting to modify an artifact (by overwriting an existing artifact of the same path with a different byte sequence), an exception is immediately thrown and artifact addition process is terminated.
>  
> This can be a bit wasteful as the server discards all other artifacts sent over the wire regardless of their own status. Thus, an improvement can be made to process all artifacts, catch any exceptions and rethrow them at the end.
> This can potentially save bandwidth by avoiding multiple transfers. Further, retry logic is unimpacted as we currently do NOT throw an error when attempting to "overwrite" an existing file with the same bytes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org