You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Brian Hulette (Jira)" <ji...@apache.org> on 2021/04/02 22:43:00 UTC
[jira] [Comment Edited] (BEAM-11777) Support correct kwargs in
aggregation methods on DataFrame, Series
[ https://issues.apache.org/jira/browse/BEAM-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314106#comment-17314106 ]
Brian Hulette edited comment on BEAM-11777 at 4/2/21, 10:42 PM:
----------------------------------------------------------------
pandas community is discussing deprecating level= on these args: https://github.com/pandas-dev/pandas/issues/39983
It's noted there that these are implemented by deferring to groupby. They actually use groupby.apply (https://github.com/pandas-dev/pandas/blob/f2c8480af2f25efdbd803218b9d87980f416563e/pandas/core/generic.py#L10562) which would be unliftable if we did it that way. But I think we could do it by just calling dropna first.
was (Author: bhulette):
pandas community is discussing deprecating level= on these args: https://github.com/pandas-dev/pandas/issues/39983
It's noted there that these are implemented by deferring to groupby. They actually use groupby.apply (https://github.com/pandas-dev/pandas/blob/f2c8480af2f25efdbd803218b9d87980f416563e/pandas/core/generic.py#L10562) which would be unliftable if we did it that way. But I think we could do it by just calling skipna first.
> Support correct kwargs in aggregation methods on DataFrame, Series
> ------------------------------------------------------------------
>
> Key: BEAM-11777
> URL: https://issues.apache.org/jira/browse/BEAM-11777
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Reporter: Brian Hulette
> Priority: P2
> Labels: dataframe-api
>
> {DataFrame,Series}.{all, any, max, min, prod, mean, median, sum} are all implemented via frame_base._agg_method, which just re-uses {DataFrame,Series}.agg}. However the pandas operations have some different kwargs that are not supported by agg. Some are universal (level=, skip_na=), others are unique to each operation (numeric_only= or bool_only=).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)