You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@heron.apache.org by Ning Wang <wa...@gmail.com> on 2018/12/17 18:29:26 UTC
12/18/2018 Bi-Weekly OSS Heron Sync-up
Hi All,
It has been two weeks since our last sync and it is time to share our
progress again. Let's share our works for the last two weeks in this
thread. And see if we need some hangout discussions or not.
My update:
- Studied the two existing reduce functions. Before implementing new
aggregation functions, I am refactoring/cleaning up the existing code and
have prepared three PRs(https://github.com/apache/incubator-heron/pull/3123,
https://github.com/apache/incubator-heron/pull/3124,
https://github.com/apache/incubator-heron/pull/3125).
Re: 12/18/2018 Bi-Weekly OSS Heron Sync-up
Posted by "J. Mark Owens" <jm...@gmail.com>.
Hi All,
I have been working to add ack/fail capability to the Java Streamlet API
for the ATLEAST_ONCE case (see issue #2909 Add ack and fail to Streamlet
API). Currently I have created acking versions of the existing Streamlet
(examples minus the Pulsar example).
I'm currently seeing the behavior I expect except in the case of the
ImpressionsAndClicks example. I'm working to track down why the ack/fail is
not working as expected in this particular case.
On Mon, Dec 17, 2018 at 1:50 PM Josh Fischer <jo...@joshfischer.io> wrote:
> I did some work to update the Heron repo to allow it to be built with the
> latest Bazel version.. I found an issue where a rule that was developed by
> the community to support Python PEX builds is causing the build to fail. I
> sent out a message to the Heron community requesting assistance with
> managing PEX through Bazel. I am still waiting to hear back from them. If
> no progress is made in the next week I will pick up work again and try to
> find a solution.
>
> On Mon, Dec 17, 2018 at 12:39 PM Bhargava Vadlamani <bh...@iu.edu>
> wrote:
>
> > Hello ,
> >
> > I finished updating the tutorial but I guess I cannot raise a PR into
> the
> > repo.
> >
> > Here are the changes I made:
> >
> > 1) Updated doc to point to latest Apache heron doc
> >
> > 2)Updated the config (pom.xml) to point to latest stable version of
> Apache
> > Heron
> >
> > 3) The old code wouldn't even compile since the method signatures and
> > actual args we pass are two different classes (like storm topology and
> > heron topology)
> >
> > I believe to the best of my knowledge that the code is good to be merged
> so
> > that new heron`ers` would take advantage of the tutorial and contribute
> if
> > they are interested
> >
> > Next steps:
> >
> > I am writing some out of the box spouts to read data from Databases so
> that
> > users can plugin heron with a database easily
> >
> > Thank you @Sree for directing me on this
> >
> > Bhargava
> >
> > On Mon, Dec 17, 2018, 10:29 AM Ning Wang <wangninggm@gmail.com wrote:
> >
> > > Hi All,
> > >
> > > It has been two weeks since our last sync and it is time to share our
> > > progress again. Let's share our works for the last two weeks in this
> > > thread. And see if we need some hangout discussions or not.
> > >
> > > My update:
> > > - Studied the two existing reduce functions. Before implementing new
> > > aggregation functions, I am refactoring/cleaning up the existing code
> and
> > > have prepared three PRs(
> > > https://github.com/apache/incubator-heron/pull/3123,
> > > https://github.com/apache/incubator-heron/pull/3124,
> > > https://github.com/apache/incubator-heron/pull/3125).
> > >
> >
>
Re: 12/18/2018 Bi-Weekly OSS Heron Sync-up
Posted by Josh Fischer <jo...@joshfischer.io>.
I did some work to update the Heron repo to allow it to be built with the
latest Bazel version.. I found an issue where a rule that was developed by
the community to support Python PEX builds is causing the build to fail. I
sent out a message to the Heron community requesting assistance with
managing PEX through Bazel. I am still waiting to hear back from them. If
no progress is made in the next week I will pick up work again and try to
find a solution.
On Mon, Dec 17, 2018 at 12:39 PM Bhargava Vadlamani <bh...@iu.edu> wrote:
> Hello ,
>
> I finished updating the tutorial but I guess I cannot raise a PR into the
> repo.
>
> Here are the changes I made:
>
> 1) Updated doc to point to latest Apache heron doc
>
> 2)Updated the config (pom.xml) to point to latest stable version of Apache
> Heron
>
> 3) The old code wouldn't even compile since the method signatures and
> actual args we pass are two different classes (like storm topology and
> heron topology)
>
> I believe to the best of my knowledge that the code is good to be merged so
> that new heron`ers` would take advantage of the tutorial and contribute if
> they are interested
>
> Next steps:
>
> I am writing some out of the box spouts to read data from Databases so that
> users can plugin heron with a database easily
>
> Thank you @Sree for directing me on this
>
> Bhargava
>
> On Mon, Dec 17, 2018, 10:29 AM Ning Wang <wangninggm@gmail.com wrote:
>
> > Hi All,
> >
> > It has been two weeks since our last sync and it is time to share our
> > progress again. Let's share our works for the last two weeks in this
> > thread. And see if we need some hangout discussions or not.
> >
> > My update:
> > - Studied the two existing reduce functions. Before implementing new
> > aggregation functions, I am refactoring/cleaning up the existing code and
> > have prepared three PRs(
> > https://github.com/apache/incubator-heron/pull/3123,
> > https://github.com/apache/incubator-heron/pull/3124,
> > https://github.com/apache/incubator-heron/pull/3125).
> >
>
Re: 12/18/2018 Bi-Weekly OSS Heron Sync-up
Posted by Ning Wang <wa...@gmail.com>.
Hi,
Bhargava, thanks for your help!
you should be able to raise a PR I believe.
1. clone the repo to your own space.
2. create your own feature branch and commit the changes.
3. git push to your repo
There should be a message in github heron repo about creating a new PR for
the change.
Please feel free to ping me if you have any questions.
On Mon, Dec 17, 2018 at 10:39 AM Bhargava Vadlamani <bh...@iu.edu> wrote:
> Hello ,
>
> I finished updating the tutorial but I guess I cannot raise a PR into the
> repo.
>
> Here are the changes I made:
>
> 1) Updated doc to point to latest Apache heron doc
>
> 2)Updated the config (pom.xml) to point to latest stable version of Apache
> Heron
>
> 3) The old code wouldn't even compile since the method signatures and
> actual args we pass are two different classes (like storm topology and
> heron topology)
>
> I believe to the best of my knowledge that the code is good to be merged so
> that new heron`ers` would take advantage of the tutorial and contribute if
> they are interested
>
> Next steps:
>
> I am writing some out of the box spouts to read data from Databases so that
> users can plugin heron with a database easily
>
> Thank you @Sree for directing me on this
>
> Bhargava
>
> On Mon, Dec 17, 2018, 10:29 AM Ning Wang <wangninggm@gmail.com wrote:
>
> > Hi All,
> >
> > It has been two weeks since our last sync and it is time to share our
> > progress again. Let's share our works for the last two weeks in this
> > thread. And see if we need some hangout discussions or not.
> >
> > My update:
> > - Studied the two existing reduce functions. Before implementing new
> > aggregation functions, I am refactoring/cleaning up the existing code and
> > have prepared three PRs(
> > https://github.com/apache/incubator-heron/pull/3123,
> > https://github.com/apache/incubator-heron/pull/3124,
> > https://github.com/apache/incubator-heron/pull/3125).
> >
>
Re: 12/18/2018 Bi-Weekly OSS Heron Sync-up
Posted by Bhargava Vadlamani <bh...@iu.edu>.
Hello ,
I finished updating the tutorial but I guess I cannot raise a PR into the
repo.
Here are the changes I made:
1) Updated doc to point to latest Apache heron doc
2)Updated the config (pom.xml) to point to latest stable version of Apache
Heron
3) The old code wouldn't even compile since the method signatures and
actual args we pass are two different classes (like storm topology and
heron topology)
I believe to the best of my knowledge that the code is good to be merged so
that new heron`ers` would take advantage of the tutorial and contribute if
they are interested
Next steps:
I am writing some out of the box spouts to read data from Databases so that
users can plugin heron with a database easily
Thank you @Sree for directing me on this
Bhargava
On Mon, Dec 17, 2018, 10:29 AM Ning Wang <wangninggm@gmail.com wrote:
> Hi All,
>
> It has been two weeks since our last sync and it is time to share our
> progress again. Let's share our works for the last two weeks in this
> thread. And see if we need some hangout discussions or not.
>
> My update:
> - Studied the two existing reduce functions. Before implementing new
> aggregation functions, I am refactoring/cleaning up the existing code and
> have prepared three PRs(
> https://github.com/apache/incubator-heron/pull/3123,
> https://github.com/apache/incubator-heron/pull/3124,
> https://github.com/apache/incubator-heron/pull/3125).
>