You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Stavros Kontopoulos <st...@lightbend.com> on 2019/09/11 13:13:02 UTC

Re: Python API for mapGroupsWithState

+1 I was looking at this today, so any idea why this was not added before?

On Sat, Aug 3, 2019 at 1:57 AM Nicholas Chammas <ni...@gmail.com>
wrote:

> Can someone succinctly describe the challenge in adding the
> `mapGroupsWithState()` API to PySpark?
>
> I was hoping for some suboptimal but nonetheless working solution to be
> available in Python, as there are with Python UDFs for example, but that
> doesn't seem to be case. The JIRA ticket for arbitrary stateful
> operations in Structured Streaming
> <https://issues.apache.org/jira/browse/SPARK-19067> doesn't give any
> indication that a Python version of the API is coming.
>
> Is this something that will likely be added in the near future, or is it a
> major undertaking? Can someone briefly describe the problem?
>
> Nick
>
>

-- 
Stavros Kontopoulos
*Principal Engineer*
*Lightbend Platform <https://www.lightbend.com/lightbend-platform>*
*mob: **+30 6977967274 <+30+6977967274>*