You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/01/16 23:57:26 UTC
[jira] [Commented] (BEAM-886) Support new DoFn in Python SDK
[ https://issues.apache.org/jira/browse/BEAM-886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15824738#comment-15824738 ]
ASF GitHub Bot commented on BEAM-886:
-------------------------------------
GitHub user sb2nov opened a pull request:
https://github.com/apache/beam/pull/1782
[BEAM-886] Implement annotation based NewDoFn
- Implement the new annotation based DoFn in python see https://s.apache.org/a-new-dofn
- Migrate all the cases of DoFn to NewDoFn
- CallableWrapperDoFn now extends NewDoFn
- Typechecking is also handled using the NewDoFn
Future Work:
- Remove OldDoFn completely from the code
- Deprecate the use of Context and pass state directly to the process method
Open question:
- This changes the semantics of process being called with a single window instead of a windowSet which is why `apache_beam.transforms.sideinputs_test:SideInputsTest.test_sliding_windows` is failing.
R: @robertwb PTAL
---
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [x] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [x] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
---
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sb2nov/incubator-beam BEAM-prototype-new-dofn
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/1782.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1782
----
commit 6d4e2539f9f2f42bff45b1861f9d3f67d915bd30
Author: Sourabh Bajaj <so...@google.com>
Date: 2017-01-16T23:50:17Z
Implement annotation based NewDoFn
----
> Support new DoFn in Python SDK
> ------------------------------
>
> Key: BEAM-886
> URL: https://issues.apache.org/jira/browse/BEAM-886
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py
> Reporter: Ahmet Altay
> Assignee: Sourabh Bajaj
> Labels: backward-incompatible, sdk-consistency
>
> Figure out what is needed for supporting new DoFns, add support and removed old DoFns.
> Related Docs from Java:
> Original Proposal email:
> https://lists.apache.org/thread.html/2abf32d528dbb64b79853552c5d10c217e2194f0685af21aeb4635dd@%3Cdev.beam.apache.org%3E
> Presentation & Doc (with short Python sections):
> https://s.apache.org/presenting-a-new-dofn
> https://s.apache.org/a-new-dofn
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)