You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by "S. Sahayaraj" <ss...@quark.com> on 2018/05/25 15:15:26 UTC

Generate Beam Pipeline from JSON

Hello,
                I would like to create the Beam pipeline (in Java) from the definitions given in JSON file. Is there any Beam Compiler available? Is any specification that guides me on how to do it?  I don't want to write the driver program for every pipeline. Please suggest.

Cheers,
S. Sahayaraj

Re: Generate Beam Pipeline from JSON

Posted by Kenneth Knowles <kl...@google.com>.
I would mention two other options to keep in mind:

 - In Python, the driver program can be tiny (much smaller than JSON
probably)
 - Beam SQL (tied to the Java SDK) is experimenting with a SQL shell/CLI.
It is not quite done, but you could follow
https://issues.apache.org/jira/browse/BEAM-3773.

Kenn

On Fri, May 25, 2018 at 8:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:

> Hi,
>
> That's on the Declarative DSLs scope: XML and JSON.
>
> It's not yet ready but it's a work in progress.
>
> You can follow:
>
> https://issues.apache.org/jira/browse/BEAM-14
>
> Regards
> JB
>
> On 25/05/2018 17:15, S. Sahayaraj wrote:
> > Hello,
> >
> >                 I would like to create the Beam pipeline (in Java) from
> > the definitions given in JSON file. Is there any Beam Compiler
> > available? Is any specification that guides me on how to do it?  I don’t
> > want to write the driver program for every pipeline. Please suggest.
> >
> >
> >
> > Cheers,
> >
> > S. Sahayaraj
> >
>
> --
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: Generate Beam Pipeline from JSON

Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi,

That's on the Declarative DSLs scope: XML and JSON.

It's not yet ready but it's a work in progress.

You can follow:

https://issues.apache.org/jira/browse/BEAM-14

Regards
JB

On 25/05/2018 17:15, S. Sahayaraj wrote:
> Hello,
> 
>                 I would like to create the Beam pipeline (in Java) from
> the definitions given in JSON file. Is there any Beam Compiler
> available? Is any specification that guides me on how to do it?  I don’t
> want to write the driver program for every pipeline. Please suggest.
> 
>  
> 
> Cheers,
> 
> S. Sahayaraj
> 

-- 
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com