You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Reynaldo Baquerizo <re...@bairesdev.com> on 2021/06/11 16:48:59 UTC

Runner support for cross-language transforms, Flink, Dataflow

Hi,

What’s the state of  runners support for cross-language transforms?
Is this roadmap up to date 
https://beam.apache.org/roadmap/connectors-multi-sdk ?

I’m looking for documentation on how to run (java pipeline calling python transforms) on Dataflow/Flink. Any reference?


—
Reynaldo

Re: Runner support for cross-language transforms, Flink, Dataflow

Posted by Chamikara Jayalath <ch...@google.com>.

On Fri, Jun 11, 2021 at 9:49 AM Reynaldo Baquerizo <
reynaldo.micheline@bairesdev.com> wrote:

>
> Hi,
>
> What’s the state of  runners support for cross-language transforms?
> Is this roadmap up to date
> https://beam.apache.org/roadmap/connectors-multi-sdk ?
>
> I’m looking for documentation on how to run (java pipeline calling python
> transforms) on Dataflow/Flink. Any reference?
>

Using Java transforms from Python pipelines is generally supported for
portable Runners and Dataflow Runner v2.
Using Python transforms from Java pipelines is experimental yet (though we
have the basic API and some tests for it).

Documentation [1] covers both. So you should be able to try things out.

Thanks,
Cham

[1]
https://beam.apache.org/documentation/programming-guide/#mulit-language-pipelines

>
> —
> Reynaldo