You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2019/09/02 00:54:00 UTC
[jira] [Work logged] (BEAM-5878) Support DoFns with Keyword-only
arguments in Python 3.
[ https://issues.apache.org/jira/browse/BEAM-5878?focusedWorklogId=305028&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305028 ]
ASF GitHub Bot logged work on BEAM-5878:
----------------------------------------
Author: ASF GitHub Bot
Created on: 02/Sep/19 00:53
Start Date: 02/Sep/19 00:53
Worklog Time Spent: 10m
Work Description: lazylynx commented on pull request #9237: [BEAM-5878] support DoFns with Keyword-only arguments
URL: https://github.com/apache/beam/pull/9237#discussion_r319783108
##########
File path: sdks/python/apache_beam/internal/pickler.py
##########
@@ -136,6 +136,32 @@ def _reject_generators(unused_pickler, unused_obj):
dill.dill.Pickler.dispatch[types.GeneratorType] = _reject_generators
+# TODO: Remove this once uqfoundation/dill#313 is fixed
+if sys.version_info[0] > 2:
+ # Monkey patch for dill._dill.Pickler to pickle functions
+ # with keyword-only args
+ _create_function = dill.dill._create_function
+
+ def _create_function_has_kwdefaults(fcode, fglobals, fname=None,
+ fdefaults=None, fclosure=None, fdict=None,
+ fkwdefaults=None):
+ func = _create_function(fcode, fglobals, fname, fdefaults, fclosure, fdict)
+ func.__kwdefaults__ = fkwdefaults
+ return func
+
+ def new_save_reduce(self, func, args, state=None, listitems=None,
Review comment:
@tvalentyn Sorry for late. PTAL
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 305028)
Time Spent: 9h 50m (was: 9h 40m)
> Support DoFns with Keyword-only arguments in Python 3.
> ------------------------------------------------------
>
> Key: BEAM-5878
> URL: https://issues.apache.org/jira/browse/BEAM-5878
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: yoshiki obata
> Priority: Minor
> Fix For: 2.16.0
>
> Time Spent: 9h 50m
> Remaining Estimate: 0h
>
> Python 3.0 [adds a possibility|https://www.python.org/dev/peps/pep-3102/] to define functions with keyword-only arguments.
> Currently Beam does not handle them correctly. [~ruoyu] pointed out [one place|https://github.com/apache/beam/blob/a56ce43109c97c739fa08adca45528c41e3c925c/sdks/python/apache_beam/typehints/decorators.py#L118] in our codebase that we should fix: in Python in 3.0 inspect.getargspec() will fail on functions with keyword-only arguments, but a new method [inspect.getfullargspec()|https://docs.python.org/3/library/inspect.html#inspect.getfullargspec] supports them.
> There may be implications for our (best-effort) type-hints machinery.
> We should also add a Py3-only unit tests that covers DoFn's with keyword-only arguments once Beam Python 3 tests are in a good shape.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)