You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/03/04 00:26:51 UTC
[GitHub] [beam] tvalentyn commented on a change in pull request #14140: [BEAM-9547] Add groupby doctests
tvalentyn commented on a change in pull request #14140:
URL: https://github.com/apache/beam/pull/14140#discussion_r586910596
##########
File path: sdks/python/apache_beam/dataframe/pandas_doctests_test.py
##########
@@ -411,6 +411,96 @@ def test_indexing_tests(self):
})
self.assertEqual(result.failed, 0)
+ def test_groupby_tests(self):
+ result = doctests.testmod(
Review comment:
1. Could you please add a docsting for `doctests.testmod()`?
For example, it's not clear what `use_beam` means and what parameters are allowed.
2. Do I understand correctly that this test will run the repl commands in pd.core.groupby.groupby (Pandas code)? If so, do we expect to be broken by Pandas adding new functionality that Beam Dataframe doesn't support yet?
##########
File path: sdks/python/apache_beam/dataframe/pandas_doctests_test.py
##########
@@ -411,6 +411,96 @@ def test_indexing_tests(self):
})
self.assertEqual(result.failed, 0)
+ def test_groupby_tests(self):
+ result = doctests.testmod(
+ pd.core.groupby.groupby,
+ use_beam=False,
+ wont_implement_ok={
+ 'pandas.core.groupby.groupby.GroupBy.head': ['*'],
+ 'pandas.core.groupby.groupby.GroupBy.tail': ['*'],
+ 'pandas.core.groupby.groupby.GroupBy.nth': ['*'],
+ 'pandas.core.groupby.groupby.GroupBy.cumcount': ['*'],
+ },
+ not_implemented_ok={
+ 'pandas.core.groupby.groupby.GroupBy.describe': ['*'],
+ 'pandas.core.groupby.groupby.GroupBy.ngroup': ['*'],
+ 'pandas.core.groupby.groupby.GroupBy.resample': ['*'],
+ 'pandas.core.groupby.groupby.GroupBy.sample': ['*'],
+ 'pandas.core.groupby.groupby.GroupBy.resample': ['*'],
+ 'pandas.core.groupby.groupby.GroupBy.quantile': ['*'],
+ 'pandas.core.groupby.groupby.BaseGroupBy.pipe': ['*'],
+ 'pandas.core.groupby.groupby.GroupBy.nth': [
+ "df.groupby('A', as_index=False).nth(1)",
+ ],
+ },
+ skip={
+ # Uses iloc to mutate a DataFrame
+ 'pandas.core.groupby.groupby.GroupBy.resample': [
+ 'df.iloc[2, 0] = 5',
+ 'df',
+ ],
+ # TODO: Raise wont implement for list passed as a grouping column
+ # Currently raises unhashable type: list
+ 'pandas.core.groupby.groupby.GroupBy.ngroup': [
+ 'df.groupby(["A", [1,1,2,3,2,1]]).ngroup()'
+ ],
+ })
+ self.assertEqual(result.failed, 0)
+
+ result = doctests.testmod(
+ pd.core.groupby.generic,
+ use_beam=False,
+ wont_implement_ok={
+ # Returns an array by default, not a Series. WontImplement
+ # (non-deferred)
+ 'pandas.core.groupby.generic.SeriesGroupBy.unique': ['*'],
+ # TODO: Is take actually deprecated?
+ 'pandas.core.groupby.generic.DataFrameGroupBy.take': ['*'],
+ 'pandas.core.groupby.generic.SeriesGroupBy.take': ['*'],
+ 'pandas.core.groupby.generic.SeriesGroupBy.nsmallest': [
+ "s.nsmallest(3, keep='last')",
Review comment:
1. Does ['*'] mean something like exclude all tests in this class?
2. I am seeing `"s.nsmallest(3, keep='last')"` in pandas/core/series.py. How does it relate to `pd.core.groupby.generic` ?
https://github.com/pandas-dev/pandas/blob/440a7de1c328aede3f566830c989922d46c254a8/pandas/core/series.py#L3686
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org