You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by Lakshmi Velineni <la...@datatorrent.com> on 2017/03/12 02:36:36 UTC
Re: example applications in malhar
Hi,
I have come up with the following list of applications to move from
datatorrent/examples into apex-malhar/examples. There were many examples
that do a similar functionality of moving data around from different
sources to destinations, so I separated them all into a separate folder
called "Ingestion".
csvformatter
dedup
dynamic-partition
enricher
exactly-once
filter
innerjoin
maprapp
parser
partition
recordreader
throttle
topnwords
transform
unifiers
*Ingestion/*
cassandrainput
cassandraoutput
fileio
fileio-multidir
fileio-simple
fileoutput
filetojdbc
hdfs2kafka
hdfs-sync
jdbcingest
jdbctijdbc
jmsActiveMQ
jmsSqs
kafka
s3-to-hdfs-sync
s3output
Let me know your thoughts.
Thanks
Lakshmi Prasanna
On Thu, Feb 23, 2017 at 10:34 AM, Amol Kekre <am...@datatorrent.com> wrote:
> Yes, should merge samples into examples. Ideally the names of the examples
> should be more descriptive in terms of "how to" as opposed to a title. PI
> demo for example has lots of ways to do compute. So it could be named as
> "pi - distributed compute" in examples. Similarly if others can be named to
> bring out features to look at from the examples, it would make examples
> more useful to readers.
>
> Thks
> Amol
>
>
>
> E:amol@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre*
>
> www.datatorrent.com | apex.apache.org
>
> *Join us at Apex Big Data World-San Jose
> <http://www.apexbigdata.com/san-jose.html>, April 4, 2017!*
> [image: http://www.apexbigdata.com/san-jose-register.html]
> <http://www.apexbigdata.com/san-jose-register.html>
>
> On Thu, Feb 23, 2017 at 10:07 AM, Sanjay Pujare <sa...@datatorrent.com>
> wrote:
>
> > + for renaming to examples. While we are at it, how about merging
> "samples"
> > also in the new "examples" ?
> >
> > On Thu, Feb 23, 2017 at 9:47 AM, Munagala Ramanath <ra...@datatorrent.com>
> > wrote:
> >
> > > +1 for renaming to "examples"
> > >
> > > Ram
> > >
> > > On Thu, Feb 23, 2017 at 9:12 AM, Lakshmi Velineni <
> > lakshmi@datatorrent.com
> > > >
> > > wrote:
> > >
> > > > I am ready to bring the examples over into the demos folder. I was
> > > > wondering if anybody has any input on Thomas's suggestion to rename
> the
> > > > demos folder to examples. I would rather do that first and then bring
> > the
> > > > examples over instead of doing it the other way around as that would
> > lead
> > > > to refactoring the new examples again.
> > > >
> > > > Thanks
> > > >
> > > > On Wed, Jan 25, 2017 at 8:12 AM, Lakshmi Velineni <
> > > lakshmi@datatorrent.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Since the examples have little history I was planning to have two
> > > > > commits for every example, one for the code as the primary author
> of
> > > > > the example and another containing pom.xml and other changes to
> make
> > > > > it work under malhar.
> > > > >
> > > > > Thanks
> > > > >
> > > > > On Wed, Nov 2, 2016 at 9:49 PM, Lakshmi Velineni
> > > > > <la...@datatorrent.com> wrote:
> > > > > > Thanks for the suggestions and I am working on the process to
> > migrate
> > > > the
> > > > > > examples with the guidelines you mentioned. I will send out a
> list
> > of
> > > > > > examples and the destination modules very soon.
> > > > > >
> > > > > >
> > > > > > On Thu, Oct 27, 2016 at 1:43 PM, Thomas Weise <
> > > thomas.weise@gmail.com>
> > > > > > wrote:
> > > > > >>
> > > > > >> Maybe a good first step would be to identify which examples to
> > bring
> > > > > over
> > > > > >> and where appropriate how to structure them in Malhar (for
> > example,
> > > I
> > > > > see
> > > > > >> multiple hdfs related apps that could go into the same Maven
> > > module).
> > > > > >>
> > > > > >>
> > > > > >> On Tue, Oct 25, 2016 at 1:00 PM, Thomas Weise <th...@apache.org>
> > > wrote:
> > > > > >>
> > > > > >> > That would be great. There are a few things to consider when
> > > working
> > > > > on
> > > > > >> > it:
> > > > > >> >
> > > > > >> > * preserve attribtion
> > > > > >> > * ensure there is a test that runs the application in the CI
> > > > > >> > * check that dependencies are compatible license
> > > > > >> > * maybe extract common boilerplate code from pom.xml
> > > > > >> >
> > > > > >> > etc.
> > > > > >> >
> > > > > >> > Existing examples are under https://github.com/apache/
> > > > > >> > apex-malhar/tree/master/demos
> > > > > >> >
> > > > > >> > Perhaps we should rename it to "examples"
> > > > > >> >
> > > > > >> > I also propose that each app has a README and we add those for
> > > > > existing
> > > > > >> > apps as well.
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > Thomas
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> > On Tue, Oct 25, 2016 at 12:49 PM, Lakshmi Velineni <
> > > > > >> > lakshmi@datatorrent.com> wrote:
> > > > > >> >
> > > > > >> >> Can i work on this?
> > > > > >> >>
> > > > > >> >> Thanks
> > > > > >> >> Lakshmi Prasanna
> > > > > >> >>
> > > > > >> >> On Mon, Sep 12, 2016 at 9:41 PM, Ashwin Chandra Putta <
> > > > > >> >> ashwinchandrap@gmail.com> wrote:
> > > > > >> >>
> > > > > >> >> > Here is the JIRA:
> > > > > >> >> > https://issues.apache.org/jira/browse/APEXMALHAR-2233
> > > > > >> >> >
> > > > > >> >> > On Tue, Sep 6, 2016 at 10:20 PM, Amol Kekre <
> > > > amol@datatorrent.com>
> > > > > >> >> wrote:
> > > > > >> >> >
> > > > > >> >> > > Good idea to consolidate them into Malhar. We should
> bring
> > in
> > > > as
> > > > > >> >> > > many
> > > > > >> >> > from
> > > > > >> >> > > this gitHub as possible.
> > > > > >> >> > >
> > > > > >> >> > > Thks
> > > > > >> >> > > Amol
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > > On Tue, Sep 6, 2016 at 6:02 PM, Thomas Weise
> > > > > >> >> > > <th...@datatorrent.com>
> > > > > >> >> > > wrote:
> > > > > >> >> > >
> > > > > >> >> > > > I'm also for consolidating these different example
> > > locations.
> > > > > We
> > > > > >> >> should
> > > > > >> >> > > > also look if all of it is still relevant.
> > > > > >> >> > > >
> > > > > >> >> > > > The stuff from the DT repository needs to be brought
> into
> > > > shape
> > > > > >> >> > > > wrt
> > > > > >> >> > > > licensing, checkstyle, CI support etc.
> > > > > >> >> > > >
> > > > > >> >> > > >
> > > > > >> >> > > > On Tue, Sep 6, 2016 at 4:34 PM, Pramod Immaneni <
> > > > > >> >> > pramod@datatorrent.com>
> > > > > >> >> > > > wrote:
> > > > > >> >> > > >
> > > > > >> >> > > > > Sounds like a good idea. How about merging demos with
> > > apps
> > > > as
> > > > > >> >> well?
> > > > > >> >> > > > >
> > > > > >> >> > > > > On Tue, Sep 6, 2016 at 4:30 PM, Ashwin Chandra Putta
> <
> > > > > >> >> > > > > ashwinchandrap@gmail.com> wrote:
> > > > > >> >> > > > >
> > > > > >> >> > > > > > Hi All,
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > We have a lot of examples for apex malhar operators
> > in
> > > > the
> > > > > >> >> > following
> > > > > >> >> > > > > > repository which resides outside of malhar.
> > > > > >> >> > > > > > https://github.com/DataTorrent/examples/tree/
> > > > > master/tutorials
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Now that it has grown quite a bit, does it make
> sense
> > > to
> > > > > >> >> > > > > > bring
> > > > > >> >> some
> > > > > >> >> > > of
> > > > > >> >> > > > > the
> > > > > >> >> > > > > > most common examples to malhar repository? Probably
> > > under
> > > > > >> >> > > > > > apps
> > > > > >> >> > > > directory?
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > That way folks looking at malhar repository will
> have
> > > > some
> > > > > >> >> samples
> > > > > >> >> > to
> > > > > >> >> > > > > look
> > > > > >> >> > > > > > at without having to search elsewhere.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > --
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Regards,
> > > > > >> >> > > > > > Ashwin.
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >> >
> > > > > >> >> >
> > > > > >> >> > --
> > > > > >> >> >
> > > > > >> >> > Regards,
> > > > > >> >> > Ashwin.
> > > > > >> >> >
> > > > > >> >>
> > > > > >> >
> > > > > >> >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > _______________________________________________________
> > >
> > > Munagala V. Ramanath
> > >
> > > Software Engineer
> > >
> > > E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
> > >
> > > www.datatorrent.com | apex.apache.org
> > >
> >
>
Re: example applications in malhar
Posted by Lakshmi Velineni <la...@datatorrent.com>.
Hi Thomas,
On Sun, Mar 12, 2017 at 10:13 AM, Thomas Weise <th...@apache.org> wrote:
> I think a flat list with good naming would make it easier to find a
> matching example.
>
I was looking at how examples are laid out in other projects like spark and
saw that there was top level classification such as r, ml, sql, streaming
etc., so I thought a top level folder, where there are lot of common themed
examples, such as datat movement, would be a good way to reduce clutter.
>
> topnwords ? There are already two related examples in Malhar: wordcount and
> twitter
>
It need not be brought over.
Can some of the examples be consolidated into a single example package,
> fileioXXX/fileoutput look like candidates.
>
I will look into it.
Thanks
Lakshmi Prasanna
>
>
> On Sat, Mar 11, 2017 at 6:36 PM, Lakshmi Velineni <lakshmi@datatorrent.com
> >
> wrote:
>
> > Hi,
> >
> > I have come up with the following list of applications to move from
> > datatorrent/examples into apex-malhar/examples. There were many examples
> > that do a similar functionality of moving data around from different
> > sources to destinations, so I separated them all into a separate folder
> > called "Ingestion".
> >
> > csvformatter
> >
> > dedup
> >
> > dynamic-partition
> >
> > enricher
> >
> > exactly-once
> >
> > filter
> >
> > innerjoin
> >
> > maprapp
> >
> > parser
> >
> > partition
> >
> > recordreader
> >
> > throttle
> >
> > topnwords
> >
> > transform
> >
> > unifiers
> >
> > *Ingestion/*
> >
> > cassandrainput
> >
> > cassandraoutput
> >
> > fileio
> >
> > fileio-multidir
> >
> > fileio-simple
> >
> > fileoutput
> >
> > filetojdbc
> >
> > hdfs2kafka
> >
> > hdfs-sync
> >
> > jdbcingest
> >
> > jdbctijdbc
> >
> > jmsActiveMQ
> >
> > jmsSqs
> >
> > kafka
> >
> > s3-to-hdfs-sync
> >
> > s3output
> >
> >
> > Let me know your thoughts.
> >
> >
> > Thanks
> >
> > Lakshmi Prasanna
> >
> >
> >
> >
> >
> >
> >
> > On Thu, Feb 23, 2017 at 10:34 AM, Amol Kekre <am...@datatorrent.com>
> wrote:
> >
> > > Yes, should merge samples into examples. Ideally the names of the
> > examples
> > > should be more descriptive in terms of "how to" as opposed to a title.
> PI
> > > demo for example has lots of ways to do compute. So it could be named
> as
> > > "pi - distributed compute" in examples. Similarly if others can be
> named
> > to
> > > bring out features to look at from the examples, it would make examples
> > > more useful to readers.
> > >
> > > Thks
> > > Amol
> > >
> > >
> > >
> > > E:amol@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre*
> > >
> > > www.datatorrent.com | apex.apache.org
> > >
> > > *Join us at Apex Big Data World-San Jose
> > > <http://www.apexbigdata.com/san-jose.html>, April 4, 2017!*
> > > [image: http://www.apexbigdata.com/san-jose-register.html]
> > > <http://www.apexbigdata.com/san-jose-register.html>
> > >
> > > On Thu, Feb 23, 2017 at 10:07 AM, Sanjay Pujare <
> sanjay@datatorrent.com>
> > > wrote:
> > >
> > > > + for renaming to examples. While we are at it, how about merging
> > > "samples"
> > > > also in the new "examples" ?
> > > >
> > > > On Thu, Feb 23, 2017 at 9:47 AM, Munagala Ramanath <
> > ram@datatorrent.com>
> > > > wrote:
> > > >
> > > > > +1 for renaming to "examples"
> > > > >
> > > > > Ram
> > > > >
> > > > > On Thu, Feb 23, 2017 at 9:12 AM, Lakshmi Velineni <
> > > > lakshmi@datatorrent.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > I am ready to bring the examples over into the demos folder. I
> was
> > > > > > wondering if anybody has any input on Thomas's suggestion to
> rename
> > > the
> > > > > > demos folder to examples. I would rather do that first and then
> > bring
> > > > the
> > > > > > examples over instead of doing it the other way around as that
> > would
> > > > lead
> > > > > > to refactoring the new examples again.
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > > On Wed, Jan 25, 2017 at 8:12 AM, Lakshmi Velineni <
> > > > > lakshmi@datatorrent.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Since the examples have little history I was planning to have
> two
> > > > > > > commits for every example, one for the code as the primary
> author
> > > of
> > > > > > > the example and another containing pom.xml and other changes to
> > > make
> > > > > > > it work under malhar.
> > > > > > >
> > > > > > > Thanks
> > > > > > >
> > > > > > > On Wed, Nov 2, 2016 at 9:49 PM, Lakshmi Velineni
> > > > > > > <la...@datatorrent.com> wrote:
> > > > > > > > Thanks for the suggestions and I am working on the process to
> > > > migrate
> > > > > > the
> > > > > > > > examples with the guidelines you mentioned. I will send out a
> > > list
> > > > of
> > > > > > > > examples and the destination modules very soon.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Oct 27, 2016 at 1:43 PM, Thomas Weise <
> > > > > thomas.weise@gmail.com>
> > > > > > > > wrote:
> > > > > > > >>
> > > > > > > >> Maybe a good first step would be to identify which examples
> to
> > > > bring
> > > > > > > over
> > > > > > > >> and where appropriate how to structure them in Malhar (for
> > > > example,
> > > > > I
> > > > > > > see
> > > > > > > >> multiple hdfs related apps that could go into the same Maven
> > > > > module).
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Tue, Oct 25, 2016 at 1:00 PM, Thomas Weise <
> thw@apache.org
> > >
> > > > > wrote:
> > > > > > > >>
> > > > > > > >> > That would be great. There are a few things to consider
> when
> > > > > working
> > > > > > > on
> > > > > > > >> > it:
> > > > > > > >> >
> > > > > > > >> > * preserve attribtion
> > > > > > > >> > * ensure there is a test that runs the application in the
> CI
> > > > > > > >> > * check that dependencies are compatible license
> > > > > > > >> > * maybe extract common boilerplate code from pom.xml
> > > > > > > >> >
> > > > > > > >> > etc.
> > > > > > > >> >
> > > > > > > >> > Existing examples are under https://github.com/apache/
> > > > > > > >> > apex-malhar/tree/master/demos
> > > > > > > >> >
> > > > > > > >> > Perhaps we should rename it to "examples"
> > > > > > > >> >
> > > > > > > >> > I also propose that each app has a README and we add those
> > for
> > > > > > > existing
> > > > > > > >> > apps as well.
> > > > > > > >> >
> > > > > > > >> > Thanks,
> > > > > > > >> > Thomas
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > On Tue, Oct 25, 2016 at 12:49 PM, Lakshmi Velineni <
> > > > > > > >> > lakshmi@datatorrent.com> wrote:
> > > > > > > >> >
> > > > > > > >> >> Can i work on this?
> > > > > > > >> >>
> > > > > > > >> >> Thanks
> > > > > > > >> >> Lakshmi Prasanna
> > > > > > > >> >>
> > > > > > > >> >> On Mon, Sep 12, 2016 at 9:41 PM, Ashwin Chandra Putta <
> > > > > > > >> >> ashwinchandrap@gmail.com> wrote:
> > > > > > > >> >>
> > > > > > > >> >> > Here is the JIRA:
> > > > > > > >> >> > https://issues.apache.org/jira/browse/APEXMALHAR-2233
> > > > > > > >> >> >
> > > > > > > >> >> > On Tue, Sep 6, 2016 at 10:20 PM, Amol Kekre <
> > > > > > amol@datatorrent.com>
> > > > > > > >> >> wrote:
> > > > > > > >> >> >
> > > > > > > >> >> > > Good idea to consolidate them into Malhar. We should
> > > bring
> > > > in
> > > > > > as
> > > > > > > >> >> > > many
> > > > > > > >> >> > from
> > > > > > > >> >> > > this gitHub as possible.
> > > > > > > >> >> > >
> > > > > > > >> >> > > Thks
> > > > > > > >> >> > > Amol
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > > On Tue, Sep 6, 2016 at 6:02 PM, Thomas Weise
> > > > > > > >> >> > > <th...@datatorrent.com>
> > > > > > > >> >> > > wrote:
> > > > > > > >> >> > >
> > > > > > > >> >> > > > I'm also for consolidating these different example
> > > > > locations.
> > > > > > > We
> > > > > > > >> >> should
> > > > > > > >> >> > > > also look if all of it is still relevant.
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > The stuff from the DT repository needs to be
> brought
> > > into
> > > > > > shape
> > > > > > > >> >> > > > wrt
> > > > > > > >> >> > > > licensing, checkstyle, CI support etc.
> > > > > > > >> >> > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > On Tue, Sep 6, 2016 at 4:34 PM, Pramod Immaneni <
> > > > > > > >> >> > pramod@datatorrent.com>
> > > > > > > >> >> > > > wrote:
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > > Sounds like a good idea. How about merging demos
> > with
> > > > > apps
> > > > > > as
> > > > > > > >> >> well?
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > On Tue, Sep 6, 2016 at 4:30 PM, Ashwin Chandra
> > Putta
> > > <
> > > > > > > >> >> > > > > ashwinchandrap@gmail.com> wrote:
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > > Hi All,
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > We have a lot of examples for apex malhar
> > operators
> > > > in
> > > > > > the
> > > > > > > >> >> > following
> > > > > > > >> >> > > > > > repository which resides outside of malhar.
> > > > > > > >> >> > > > > > https://github.com/DataTorrent/examples/tree/
> > > > > > > master/tutorials
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Now that it has grown quite a bit, does it make
> > > sense
> > > > > to
> > > > > > > >> >> > > > > > bring
> > > > > > > >> >> some
> > > > > > > >> >> > > of
> > > > > > > >> >> > > > > the
> > > > > > > >> >> > > > > > most common examples to malhar repository?
> > Probably
> > > > > under
> > > > > > > >> >> > > > > > apps
> > > > > > > >> >> > > > directory?
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > That way folks looking at malhar repository
> will
> > > have
> > > > > > some
> > > > > > > >> >> samples
> > > > > > > >> >> > to
> > > > > > > >> >> > > > > look
> > > > > > > >> >> > > > > > at without having to search elsewhere.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > --
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Regards,
> > > > > > > >> >> > > > > > Ashwin.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >> >
> > > > > > > >> >> >
> > > > > > > >> >> > --
> > > > > > > >> >> >
> > > > > > > >> >> > Regards,
> > > > > > > >> >> > Ashwin.
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > _______________________________________________________
> > > > >
> > > > > Munagala V. Ramanath
> > > > >
> > > > > Software Engineer
> > > > >
> > > > > E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
> > > > >
> > > > > www.datatorrent.com | apex.apache.org
> > > > >
> > > >
> > >
> >
>
Re: example applications in malhar
Posted by Thomas Weise <th...@apache.org>.
I think a flat list with good naming would make it easier to find a
matching example.
topnwords ? There are already two related examples in Malhar: wordcount and
twitter
Can some of the examples be consolidated into a single example package,
fileioXXX/fileoutput look like candidates.
On Sat, Mar 11, 2017 at 6:36 PM, Lakshmi Velineni <la...@datatorrent.com>
wrote:
> Hi,
>
> I have come up with the following list of applications to move from
> datatorrent/examples into apex-malhar/examples. There were many examples
> that do a similar functionality of moving data around from different
> sources to destinations, so I separated them all into a separate folder
> called "Ingestion".
>
> csvformatter
>
> dedup
>
> dynamic-partition
>
> enricher
>
> exactly-once
>
> filter
>
> innerjoin
>
> maprapp
>
> parser
>
> partition
>
> recordreader
>
> throttle
>
> topnwords
>
> transform
>
> unifiers
>
> *Ingestion/*
>
> cassandrainput
>
> cassandraoutput
>
> fileio
>
> fileio-multidir
>
> fileio-simple
>
> fileoutput
>
> filetojdbc
>
> hdfs2kafka
>
> hdfs-sync
>
> jdbcingest
>
> jdbctijdbc
>
> jmsActiveMQ
>
> jmsSqs
>
> kafka
>
> s3-to-hdfs-sync
>
> s3output
>
>
> Let me know your thoughts.
>
>
> Thanks
>
> Lakshmi Prasanna
>
>
>
>
>
>
>
> On Thu, Feb 23, 2017 at 10:34 AM, Amol Kekre <am...@datatorrent.com> wrote:
>
> > Yes, should merge samples into examples. Ideally the names of the
> examples
> > should be more descriptive in terms of "how to" as opposed to a title. PI
> > demo for example has lots of ways to do compute. So it could be named as
> > "pi - distributed compute" in examples. Similarly if others can be named
> to
> > bring out features to look at from the examples, it would make examples
> > more useful to readers.
> >
> > Thks
> > Amol
> >
> >
> >
> > E:amol@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre*
> >
> > www.datatorrent.com | apex.apache.org
> >
> > *Join us at Apex Big Data World-San Jose
> > <http://www.apexbigdata.com/san-jose.html>, April 4, 2017!*
> > [image: http://www.apexbigdata.com/san-jose-register.html]
> > <http://www.apexbigdata.com/san-jose-register.html>
> >
> > On Thu, Feb 23, 2017 at 10:07 AM, Sanjay Pujare <sa...@datatorrent.com>
> > wrote:
> >
> > > + for renaming to examples. While we are at it, how about merging
> > "samples"
> > > also in the new "examples" ?
> > >
> > > On Thu, Feb 23, 2017 at 9:47 AM, Munagala Ramanath <
> ram@datatorrent.com>
> > > wrote:
> > >
> > > > +1 for renaming to "examples"
> > > >
> > > > Ram
> > > >
> > > > On Thu, Feb 23, 2017 at 9:12 AM, Lakshmi Velineni <
> > > lakshmi@datatorrent.com
> > > > >
> > > > wrote:
> > > >
> > > > > I am ready to bring the examples over into the demos folder. I was
> > > > > wondering if anybody has any input on Thomas's suggestion to rename
> > the
> > > > > demos folder to examples. I would rather do that first and then
> bring
> > > the
> > > > > examples over instead of doing it the other way around as that
> would
> > > lead
> > > > > to refactoring the new examples again.
> > > > >
> > > > > Thanks
> > > > >
> > > > > On Wed, Jan 25, 2017 at 8:12 AM, Lakshmi Velineni <
> > > > lakshmi@datatorrent.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Since the examples have little history I was planning to have two
> > > > > > commits for every example, one for the code as the primary author
> > of
> > > > > > the example and another containing pom.xml and other changes to
> > make
> > > > > > it work under malhar.
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > > On Wed, Nov 2, 2016 at 9:49 PM, Lakshmi Velineni
> > > > > > <la...@datatorrent.com> wrote:
> > > > > > > Thanks for the suggestions and I am working on the process to
> > > migrate
> > > > > the
> > > > > > > examples with the guidelines you mentioned. I will send out a
> > list
> > > of
> > > > > > > examples and the destination modules very soon.
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Oct 27, 2016 at 1:43 PM, Thomas Weise <
> > > > thomas.weise@gmail.com>
> > > > > > > wrote:
> > > > > > >>
> > > > > > >> Maybe a good first step would be to identify which examples to
> > > bring
> > > > > > over
> > > > > > >> and where appropriate how to structure them in Malhar (for
> > > example,
> > > > I
> > > > > > see
> > > > > > >> multiple hdfs related apps that could go into the same Maven
> > > > module).
> > > > > > >>
> > > > > > >>
> > > > > > >> On Tue, Oct 25, 2016 at 1:00 PM, Thomas Weise <thw@apache.org
> >
> > > > wrote:
> > > > > > >>
> > > > > > >> > That would be great. There are a few things to consider when
> > > > working
> > > > > > on
> > > > > > >> > it:
> > > > > > >> >
> > > > > > >> > * preserve attribtion
> > > > > > >> > * ensure there is a test that runs the application in the CI
> > > > > > >> > * check that dependencies are compatible license
> > > > > > >> > * maybe extract common boilerplate code from pom.xml
> > > > > > >> >
> > > > > > >> > etc.
> > > > > > >> >
> > > > > > >> > Existing examples are under https://github.com/apache/
> > > > > > >> > apex-malhar/tree/master/demos
> > > > > > >> >
> > > > > > >> > Perhaps we should rename it to "examples"
> > > > > > >> >
> > > > > > >> > I also propose that each app has a README and we add those
> for
> > > > > > existing
> > > > > > >> > apps as well.
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Thomas
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > On Tue, Oct 25, 2016 at 12:49 PM, Lakshmi Velineni <
> > > > > > >> > lakshmi@datatorrent.com> wrote:
> > > > > > >> >
> > > > > > >> >> Can i work on this?
> > > > > > >> >>
> > > > > > >> >> Thanks
> > > > > > >> >> Lakshmi Prasanna
> > > > > > >> >>
> > > > > > >> >> On Mon, Sep 12, 2016 at 9:41 PM, Ashwin Chandra Putta <
> > > > > > >> >> ashwinchandrap@gmail.com> wrote:
> > > > > > >> >>
> > > > > > >> >> > Here is the JIRA:
> > > > > > >> >> > https://issues.apache.org/jira/browse/APEXMALHAR-2233
> > > > > > >> >> >
> > > > > > >> >> > On Tue, Sep 6, 2016 at 10:20 PM, Amol Kekre <
> > > > > amol@datatorrent.com>
> > > > > > >> >> wrote:
> > > > > > >> >> >
> > > > > > >> >> > > Good idea to consolidate them into Malhar. We should
> > bring
> > > in
> > > > > as
> > > > > > >> >> > > many
> > > > > > >> >> > from
> > > > > > >> >> > > this gitHub as possible.
> > > > > > >> >> > >
> > > > > > >> >> > > Thks
> > > > > > >> >> > > Amol
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > On Tue, Sep 6, 2016 at 6:02 PM, Thomas Weise
> > > > > > >> >> > > <th...@datatorrent.com>
> > > > > > >> >> > > wrote:
> > > > > > >> >> > >
> > > > > > >> >> > > > I'm also for consolidating these different example
> > > > locations.
> > > > > > We
> > > > > > >> >> should
> > > > > > >> >> > > > also look if all of it is still relevant.
> > > > > > >> >> > > >
> > > > > > >> >> > > > The stuff from the DT repository needs to be brought
> > into
> > > > > shape
> > > > > > >> >> > > > wrt
> > > > > > >> >> > > > licensing, checkstyle, CI support etc.
> > > > > > >> >> > > >
> > > > > > >> >> > > >
> > > > > > >> >> > > > On Tue, Sep 6, 2016 at 4:34 PM, Pramod Immaneni <
> > > > > > >> >> > pramod@datatorrent.com>
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > >
> > > > > > >> >> > > > > Sounds like a good idea. How about merging demos
> with
> > > > apps
> > > > > as
> > > > > > >> >> well?
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > On Tue, Sep 6, 2016 at 4:30 PM, Ashwin Chandra
> Putta
> > <
> > > > > > >> >> > > > > ashwinchandrap@gmail.com> wrote:
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > > Hi All,
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > We have a lot of examples for apex malhar
> operators
> > > in
> > > > > the
> > > > > > >> >> > following
> > > > > > >> >> > > > > > repository which resides outside of malhar.
> > > > > > >> >> > > > > > https://github.com/DataTorrent/examples/tree/
> > > > > > master/tutorials
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Now that it has grown quite a bit, does it make
> > sense
> > > > to
> > > > > > >> >> > > > > > bring
> > > > > > >> >> some
> > > > > > >> >> > > of
> > > > > > >> >> > > > > the
> > > > > > >> >> > > > > > most common examples to malhar repository?
> Probably
> > > > under
> > > > > > >> >> > > > > > apps
> > > > > > >> >> > > > directory?
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > That way folks looking at malhar repository will
> > have
> > > > > some
> > > > > > >> >> samples
> > > > > > >> >> > to
> > > > > > >> >> > > > > look
> > > > > > >> >> > > > > > at without having to search elsewhere.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > --
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Regards,
> > > > > > >> >> > > > > > Ashwin.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >> >
> > > > > > >> >> >
> > > > > > >> >> > --
> > > > > > >> >> >
> > > > > > >> >> > Regards,
> > > > > > >> >> > Ashwin.
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >> >
> > > > > > >> >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > _______________________________________________________
> > > >
> > > > Munagala V. Ramanath
> > > >
> > > > Software Engineer
> > > >
> > > > E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
> > > >
> > > > www.datatorrent.com | apex.apache.org
> > > >
> > >
> >
>