You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Livia Zhu (Jira)" <ji...@apache.org> on 2025/04/25 21:52:00 UTC

[jira] [Created] (SPARK-51922) Fix UTFDataFormatException thrown from StateStoreChangelogReaderFactory for v1

Livia Zhu created SPARK-51922:
---------------------------------

             Summary: Fix UTFDataFormatException thrown from StateStoreChangelogReaderFactory for v1
                 Key: SPARK-51922
                 URL: https://issues.apache.org/jira/browse/SPARK-51922
             Project: Spark
          Issue Type: Task
          Components: Structured Streaming
    Affects Versions: 4.0.0
            Reporter: Livia Zhu


As part of StateStoreChangelogReaderFactory, we try to read the file using readUTF to get the version string. However, v1 does not write the version string to the file as a UTF and instead the first few bytes are an Int. For a sufficiently large key, this will throw a UTFDataFormatException. We should catch this exception so it doesn't fail the query.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org