You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@beam.apache.org by Henning Rohde <he...@google.com.INVALID> on 2017/09/08 20:59:36 UTC

Portability JIRA organization

Hi everyone,

 The portability effort involves a large amount of work cutting across all
SDKs and runners [e..g, 1,2,3,4]. It seems to be only partially captured in
JIRAs, so I'd like to volunteer to try to flesh it out further. In addition
to adding JIRAs for missing work, I was thinking of organizing it as
follows:

  (1) Designs, proto definitions and the like belong in the "beam-model"
component. For example, BEAM-2862 tracks support for User State over the Fn
API.
  (2) Dependent work are subtasks belong in the component where the work is
needed. For example, making the Python SDK harness use the User State over
the Fn API would be in the "sdk-py" component and be a subtask (or
otherwise linked to) the overall BEAM-2862 issue. Similarly for each runner
and other SDKs.
  (3) All portability issues should use a "portability" label, regardless
of component, to identify the overall effort.

The aim is to make it more clear to see what works (and remains to be done)
from both the point of view of each SDK and runner as well as feature-wise
for portability.

Please let me know if you have any comments or objections.

Thanks,
 Henning

[1] https://s.apache.org/beam-fn-api
[2] https://s.apache.org/beam-runner-api
[3] https://s.apache.org/beam-job-api
[4] https://s.apache.org/beam-fn-api-container-contract

Re: Portability JIRA organization

Posted by Henning Rohde <he...@google.com.INVALID>.

Thanks Kenn and Aljoscha! I've added a bunch of issues and reorganized some
of existing ones as per the proposal. Of course, anyone should feel free to
make changes if something is off or doesn't make sense.

I would like to call out a few of the new issues:

   * BEAM-2899: "Universal Local Runner".
   * BEAM-2896: "Portability milestone: wordcount runs everywhere".
   * BEAM-2941: "Portability milestone: windowed wordcount runs everywhere".
   * BEAM-2940: "Portability milestone: mobile gaming runs everywhere".

The first is a local reference implementation for the portability API. The
latter three are intended to track the bigger milestones that all SDKs and
runners implement enough of the portability framework to run X.

Thanks,
 Henning



On Fri, Sep 8, 2017 at 4:42 PM, Aljoscha Krettek <al...@apache.org>
wrote:

> +1
>
> Sounds great!
>
> > On 9. Sep 2017, at 00:44, Kenneth Knowles <kl...@google.com.INVALID>
> wrote:
> >
> > +1 to this.
> >
> > It is really easy to lose track of things in a sea of tickets, and
> > portability touches every SDK and runner, so getting this organized will
> be
> > hugely helpful. Especially getting some clear higher-level milestones (by
> > moving tickets to subtasks) we can work towards and announce when done
> will
> > be great.
> >
> > Kenn
> >
> > On Fri, Sep 8, 2017 at 1:59 PM, Henning Rohde <herohde@google.com.invalid
> >
> > wrote:
> >
> >> Hi everyone,
> >>
> >> The portability effort involves a large amount of work cutting across
> all
> >> SDKs and runners [e..g, 1,2,3,4]. It seems to be only partially
> captured in
> >> JIRAs, so I'd like to volunteer to try to flesh it out further. In
> addition
> >> to adding JIRAs for missing work, I was thinking of organizing it as
> >> follows:
> >>
> >>  (1) Designs, proto definitions and the like belong in the "beam-model"
> >> component. For example, BEAM-2862 tracks support for User State over
> the Fn
> >> API.
> >>  (2) Dependent work are subtasks belong in the component where the work
> is
> >> needed. For example, making the Python SDK harness use the User State
> over
> >> the Fn API would be in the "sdk-py" component and be a subtask (or
> >> otherwise linked to) the overall BEAM-2862 issue. Similarly for each
> runner
> >> and other SDKs.
> >>  (3) All portability issues should use a "portability" label, regardless
> >> of component, to identify the overall effort.
> >>
> >> The aim is to make it more clear to see what works (and remains to be
> done)
> >> from both the point of view of each SDK and runner as well as
> feature-wise
> >> for portability.
> >>
> >> Please let me know if you have any comments or objections.
> >>
> >> Thanks,
> >> Henning
> >>
> >> [1] https://s.apache.org/beam-fn-api
> >> [2] https://s.apache.org/beam-runner-api
> >> [3] https://s.apache.org/beam-job-api
> >> [4] https://s.apache.org/beam-fn-api-container-contract
> >>
>
>

Re: Portability JIRA organization

Posted by Aljoscha Krettek <al...@apache.org>.

+1

Sounds great!

> On 9. Sep 2017, at 00:44, Kenneth Knowles <kl...@google.com.INVALID> wrote:
> 
> +1 to this.
> 
> It is really easy to lose track of things in a sea of tickets, and
> portability touches every SDK and runner, so getting this organized will be
> hugely helpful. Especially getting some clear higher-level milestones (by
> moving tickets to subtasks) we can work towards and announce when done will
> be great.
> 
> Kenn
> 
> On Fri, Sep 8, 2017 at 1:59 PM, Henning Rohde <he...@google.com.invalid>
> wrote:
> 
>> Hi everyone,
>> 
>> The portability effort involves a large amount of work cutting across all
>> SDKs and runners [e..g, 1,2,3,4]. It seems to be only partially captured in
>> JIRAs, so I'd like to volunteer to try to flesh it out further. In addition
>> to adding JIRAs for missing work, I was thinking of organizing it as
>> follows:
>> 
>>  (1) Designs, proto definitions and the like belong in the "beam-model"
>> component. For example, BEAM-2862 tracks support for User State over the Fn
>> API.
>>  (2) Dependent work are subtasks belong in the component where the work is
>> needed. For example, making the Python SDK harness use the User State over
>> the Fn API would be in the "sdk-py" component and be a subtask (or
>> otherwise linked to) the overall BEAM-2862 issue. Similarly for each runner
>> and other SDKs.
>>  (3) All portability issues should use a "portability" label, regardless
>> of component, to identify the overall effort.
>> 
>> The aim is to make it more clear to see what works (and remains to be done)
>> from both the point of view of each SDK and runner as well as feature-wise
>> for portability.
>> 
>> Please let me know if you have any comments or objections.
>> 
>> Thanks,
>> Henning
>> 
>> [1] https://s.apache.org/beam-fn-api
>> [2] https://s.apache.org/beam-runner-api
>> [3] https://s.apache.org/beam-job-api
>> [4] https://s.apache.org/beam-fn-api-container-contract
>>

Re: Portability JIRA organization

Posted by Kenneth Knowles <kl...@google.com.INVALID>.

+1 to this.

It is really easy to lose track of things in a sea of tickets, and
portability touches every SDK and runner, so getting this organized will be
hugely helpful. Especially getting some clear higher-level milestones (by
moving tickets to subtasks) we can work towards and announce when done will
be great.

Kenn

On Fri, Sep 8, 2017 at 1:59 PM, Henning Rohde <he...@google.com.invalid>
wrote:

> Hi everyone,
>
>  The portability effort involves a large amount of work cutting across all
> SDKs and runners [e..g, 1,2,3,4]. It seems to be only partially captured in
> JIRAs, so I'd like to volunteer to try to flesh it out further. In addition
> to adding JIRAs for missing work, I was thinking of organizing it as
> follows:
>
>   (1) Designs, proto definitions and the like belong in the "beam-model"
> component. For example, BEAM-2862 tracks support for User State over the Fn
> API.
>   (2) Dependent work are subtasks belong in the component where the work is
> needed. For example, making the Python SDK harness use the User State over
> the Fn API would be in the "sdk-py" component and be a subtask (or
> otherwise linked to) the overall BEAM-2862 issue. Similarly for each runner
> and other SDKs.
>   (3) All portability issues should use a "portability" label, regardless
> of component, to identify the overall effort.
>
> The aim is to make it more clear to see what works (and remains to be done)
> from both the point of view of each SDK and runner as well as feature-wise
> for portability.
>
> Please let me know if you have any comments or objections.
>
> Thanks,
>  Henning
>
> [1] https://s.apache.org/beam-fn-api
> [2] https://s.apache.org/beam-runner-api
> [3] https://s.apache.org/beam-job-api
> [4] https://s.apache.org/beam-fn-api-container-contract
>