Can we have more quick start examples ?

[Manu Zhang <ow...@gmail.com>]

Hey guys,

I find Beam examples under the examples folder are not easy to run due to
dependency on Google specific services. Even the MinimalWordCount
<https://github.com/apache/incubator-beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java>
requires
input and output to be on Google Cloud Storage. Others like
WindowedWordCount
<https://github.com/apache/incubator-beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/WindowedWordCount.java>
require
BigQuery.  I wouldn't expect newcomers to tweak IO themselves.

Can we have more quick start examples that can be run anywhere ?

Thanks,
Manu Zhang

Re: Can we have more quick start examples ?

[Neelesh Salian <ns...@cloudera.com>]

+1 to this.
I liked the guides for the setup for GC and Storage.
The documentation is by far better than any guide I have seen. I also
provided feedback on the documentation where it could use improvement.

But certainly a more abstract and user friendly example would be
encouraging for new users and folks curious in using Beam itself.

On Thu, Oct 27, 2016 at 11:49 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:

> Yes it sounds good to me. I would love to see this as part of the examples.
>
> Ismael and I also started the beam-samples (http://github.com/jbonofre/
> beam-samples) that could be part of the examples.
> The purpose is to have more real use cases implementation with real data.
>
> Regards
> JB
>
> ⁣​
>
> On Oct 27, 2016, 17:17, at 17:17, Jesse Anderson <je...@smokinghand.com>
> wrote:
> >
>



-- 
Neelesh Srinivas Salian
Customer Operations Engineer

Re: Can we have more quick start examples ?

[Jean-Baptiste Onofré <jb...@nanthrax.net>]

Yes it sounds good to me. I would love to see this as part of the examples.

Ismael and I also started the beam-samples (http://github.com/jbonofre/beam-samples) that could be part of the examples.
The purpose is to have more real use cases implementation with real data.

Regards
JB

\u2063\u200b

On Oct 27, 2016, 17:17, at 17:17, Jesse Anderson <je...@smokinghand.com> wrote:
>

Re: Can we have more quick start examples ?

[Davor Bonaci <da...@apache.org>]

Indeed -- this is a clear area for improvement. Sources are usually not as
big of an issue -- these resources are publicly accessible regardless
where/how you run the pipeline (locally, or with any runner). On the other
hand, Sinks require write access, which is often more problematic.

One correction, however: WordCount supports both GCS and local paths, with
some exceptions depending on a runner.

There are several efforts to improve this, most notably BEAM-59, which is
assigned to Pei.

On Thu, Oct 27, 2016 at 8:17 AM, Jesse Anderson <je...@smokinghand.com>
wrote:

> Those tutorials help. I was going through the example code and had the same
> thought. We need to take a pass through the examples and remove some of the
> Google Cloud dependencies.
>
> On Thu, Oct 27, 2016, 5:13 PM Thomas Weise <th...@apache.org> wrote:
>
> > The Beam tutorials seem to address this:
> >
> > https://github.com/eljefe6a/beamexample/blob/master/README.md
> >
> >
> > On Thu, Oct 27, 2016 at 8:04 AM, Manu Zhang <ow...@gmail.com>
> > wrote:
> >
> > > Hey guys,
> > >
> > > I find Beam examples under the examples folder are not easy to run due
> to
> > > dependency on Google specific services. Even the MinimalWordCount
> > > <https://github.com/apache/incubator-beam/blob/master/
> > >
> > examples/java/src/main/java/org/apache/beam/examples/
> MinimalWordCount.java
> > > >
> > > requires
> > > input and output to be on Google Cloud Storage. Others like
> > > WindowedWordCount
> > > <https://github.com/apache/incubator-beam/blob/master/
> > > examples/java/src/main/java/org/apache/beam/examples/
> > > WindowedWordCount.java>
> > > require
> > > BigQuery.  I wouldn't expect newcomers to tweak IO themselves.
> > >
> > > Can we have more quick start examples that can be run anywhere ?
> > >
> > > Thanks,
> > > Manu Zhang
> > >
> >
>

Re: Can we have more quick start examples ?

[Jesse Anderson <je...@smokinghand.com>]

Those tutorials help. I was going through the example code and had the same
thought. We need to take a pass through the examples and remove some of the
Google Cloud dependencies.

On Thu, Oct 27, 2016, 5:13 PM Thomas Weise <th...@apache.org> wrote:

> The Beam tutorials seem to address this:
>
> https://github.com/eljefe6a/beamexample/blob/master/README.md
>
>
> On Thu, Oct 27, 2016 at 8:04 AM, Manu Zhang <ow...@gmail.com>
> wrote:
>
> > Hey guys,
> >
> > I find Beam examples under the examples folder are not easy to run due to
> > dependency on Google specific services. Even the MinimalWordCount
> > <https://github.com/apache/incubator-beam/blob/master/
> >
> examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java
> > >
> > requires
> > input and output to be on Google Cloud Storage. Others like
> > WindowedWordCount
> > <https://github.com/apache/incubator-beam/blob/master/
> > examples/java/src/main/java/org/apache/beam/examples/
> > WindowedWordCount.java>
> > require
> > BigQuery.  I wouldn't expect newcomers to tweak IO themselves.
> >
> > Can we have more quick start examples that can be run anywhere ?
> >
> > Thanks,
> > Manu Zhang
> >
>

Re: Can we have more quick start examples ?

[Thomas Weise <th...@apache.org>]

The Beam tutorials seem to address this:

https://github.com/eljefe6a/beamexample/blob/master/README.md


On Thu, Oct 27, 2016 at 8:04 AM, Manu Zhang <ow...@gmail.com> wrote:

> Hey guys,
>
> I find Beam examples under the examples folder are not easy to run due to
> dependency on Google specific services. Even the MinimalWordCount
> <https://github.com/apache/incubator-beam/blob/master/
> examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java
> >
> requires
> input and output to be on Google Cloud Storage. Others like
> WindowedWordCount
> <https://github.com/apache/incubator-beam/blob/master/
> examples/java/src/main/java/org/apache/beam/examples/
> WindowedWordCount.java>
> require
> BigQuery.  I wouldn't expect newcomers to tweak IO themselves.
>
> Can we have more quick start examples that can be run anywhere ?
>
> Thanks,
> Manu Zhang
>