You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Livia Zhu (Jira)" <ji...@apache.org> on 2025/04/25 21:52:00 UTC
[jira] [Created] (SPARK-51922) Fix UTFDataFormatException thrown from StateStoreChangelogReaderFactory for v1
Livia Zhu created SPARK-51922:
---------------------------------
Summary: Fix UTFDataFormatException thrown from StateStoreChangelogReaderFactory for v1
Key: SPARK-51922
URL: https://issues.apache.org/jira/browse/SPARK-51922
Project: Spark
Issue Type: Task
Components: Structured Streaming
Affects Versions: 4.0.0
Reporter: Livia Zhu
As part of StateStoreChangelogReaderFactory, we try to read the file using readUTF to get the version string. However, v1 does not write the version string to the file as a UTF and instead the first few bytes are an Int. For a sufficiently large key, this will throw a UTFDataFormatException. We should catch this exception so it doesn't fail the query.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org