[jira] [Commented] (BEAM-265) Dynamic Pipeline, Transformation, etc. / DAG Changes DSL | Full Specification Language Independence, Transo

["Davor Bonaci (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275685#comment-15275685 ] 

Davor Bonaci commented on BEAM-265:
-----------------------------------

There are many things conflated here ;)

Are you referring to data-dependent pipeline structure? If so, we fully agree. This is on the wider roadmap, but not being worked on currently.

In terms of language-independence, Runner API and Fn API work currently in progress achieves that goal. (There's never full language-independence -- user-defined functions are always language-specific.)

Do you agree, or am I missing something?

> Dynamic Pipeline, Transformation, etc. / DAG Changes DSL | Full Specification Language Independence, Transo
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: BEAM-265
>                 URL: https://issues.apache.org/jira/browse/BEAM-265
>             Project: Beam
>          Issue Type: Improvement
>          Components: beam-model
>            Reporter: Suminda Dharmasena
>            Assignee: Davor Bonaci
>
> There should conditional / dynamic changes to pipelines and DAGs. The Beam specification should ideally be:
>  - fully language neutral to support many languages
>  - should be specified statically so code generation can work. E.g. pipeline specified in Java but run as native. In this case the conversion would be like that GWT does with Java.
> In addition the transformation code though expressed in a given language should be transformable into an language independent form.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)