[jira] [Commented] (BEAM-12181) Implement parallelized (approximate) mode

["Brian Hulette (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499247#comment-17499247 ] 

Brian Hulette commented on BEAM-12181:
--------------------------------------

> Could you elaborate on why it may be necessary to create our own implementation of kde()? Is the scipy impl incompatible with the series format?

Sorry seems like I wasn't clear here. What I meant was we might consider adding a DeferredSeries.kde() method, even though pandas.Series does not have a kde method. Under the hood it could certainly defer to scipy.gaussian_kde for computing partial results on the workers. The only reason to do it would be for separation of concerns - I thought it might be more straightforward to first add a distributed kde implementation that gets the same results as (non-distributed) scipy.gaussian_kde. Then we could add an approximate mode option on top of that.

> Implement parallelized (approximate) mode
> -----------------------------------------
>
>                 Key: BEAM-12181
>                 URL: https://issues.apache.org/jira/browse/BEAM-12181
>             Project: Beam
>          Issue Type: Improvement
>          Components: dsl-dataframe, sdk-py-core
>            Reporter: Brian Hulette
>            Priority: P3
>              Labels: dataframe-api
>
> Currently we require Singleton partitioning to compute mode(). We should provide an option to compute approximate mode() which can be parallelized.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)