You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2022/11/25 07:02:00 UTC

[jira] [Commented] (SPARK-41261) applyInPandasWithState can produce incorrect key value in user function for timed out state

    [ https://issues.apache.org/jira/browse/SPARK-41261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17638529#comment-17638529 ] 

Jungtaek Lim commented on SPARK-41261:
--------------------------------------

Will submit a PR shortly.

> applyInPandasWithState can produce incorrect key value in user function for timed out state
> -------------------------------------------------------------------------------------------
>
>                 Key: SPARK-41261
>                 URL: https://issues.apache.org/jira/browse/SPARK-41261
>             Project: Spark
>          Issue Type: Bug
>          Components: Structured Streaming
>    Affects Versions: 3.4.0
>            Reporter: Jungtaek Lim
>            Priority: Major
>
> We observed the issue that user function retrieves incorrect key in user function for timed out state. After RCA we figured out this could happen when the columns of grouping keys are not placed sequentially at earliest place.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org