You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2025/11/14 17:04:00 UTC

[jira] [Updated] (SPARK-54358) Checkpoint dirs collide when streaming tables in different schemas have same name

     [ https://issues.apache.org/jira/browse/SPARK-54358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-54358:
-----------------------------------
    Labels: pull-request-available  (was: )

> Checkpoint dirs collide when streaming tables in different schemas have same name
> ---------------------------------------------------------------------------------
>
>                 Key: SPARK-54358
>                 URL: https://issues.apache.org/jira/browse/SPARK-54358
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Declarative Pipelines
>    Affects Versions: 4.1.0
>            Reporter: Sanford Ryza
>            Priority: Major
>              Labels: pull-request-available
>
> A streaming table is a table fed by a stream. Streaming tables have checkpoint directories underneath their pipeline's storage root. These directories don't currently take the table namespace into account, which means that two tables with different schemas but the same name will be mapped to the same checkpoint directory. This could be very bad and cause data loss.
> We should update the per-streaming table checkpoint path to use the fully qualified table path, instead of just its name.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org