You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2024/10/18 20:28:00 UTC

[jira] [Updated] (SPARK-50017) Support Avro encoding for TransformWithState operator - ValueState

     [ https://issues.apache.org/jira/browse/SPARK-50017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-50017:
-----------------------------------
    Labels: pull-request-available  (was: )

> Support Avro encoding for TransformWithState operator - ValueState
> ------------------------------------------------------------------
>
>                 Key: SPARK-50017
>                 URL: https://issues.apache.org/jira/browse/SPARK-50017
>             Project: Spark
>          Issue Type: New Feature
>          Components: Structured Streaming
>    Affects Versions: 4.0.0
>            Reporter: Eric Marnadi
>            Priority: Major
>              Labels: pull-request-available
>
> Currently, we use UnsafeRow to store all state in the StateStores. This change will add StateStore APIs that support byte array operations on the StateStore, and to enable Avro encoding to be used for the TransformWithState operator, only for ValueState. Subsequent state variable types will be supported in future PRs. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org