You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@heron.apache.org by Thomas Cooper <to...@gmail.com> on 2018/04/01 22:02:23 UTC
Re: Stateful Updates
Hi all,
Hopefully the share permissions are correct on the doc now:
<goog_1394670011>
https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwti7Q0hmW6lmgyk8/edit?usp=sharing
Look forward to hearing what you think.
Also is it worth making an issue for this?
Cheers,
Tom Cooper
W: www.tomcooper.org.uk | Twitter: @tomncooper
<https://twitter.com/tomncooper>
On 30 March 2018 at 20:05, Karthik Ramasamy <kr...@gmail.com> wrote:
> Thanks Tom. It will be a great addition. Unable to access the doc. Could
> you give permission?
>
> Sent from my iPhone
>
> > On Mar 30, 2018, at 6:08 PM, Thomas Cooper <to...@gmail.com>
> wrote:
> >
> > Hi all,
> >
> > After chatting to the Heron Team at Twitter and Microsoft I have written
> up
> > a proposal for giving Heron the ability to preserve state across a
> topology
> > update (change in component parallelism). This is problem for components
> > with incoming fields based connections as keys will not hash to the same
> > task ID after an update. I have defined the problem (as I see it) and
> > propsoed two solutions in the google doc below:
> >
> > https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwt
> i7Q0hmW6lmgyk8/edit?usp=sharing
> >
> > I would love to get feedback on these proposals. Feel free to blow holes
> in
> > my assumptions and correct any misinterpretations I have made!
> >
> > Thanks,
> >
> > Tom Cooper
> > W: www.tomcooper.org.uk | Twitter: @tomncooper
> > <https://twitter.com/tomncooper>
>
Re: Stateful Updates
Posted by Ning Wang <wa...@gmail.com>.
I think it is a draft currently and it might be easier to collaborate in
Google Doc.
On Sun, Apr 1, 2018 at 10:49 PM, Dave Fisher <da...@comcast.net> wrote:
> Hi -
>
> Any reason this document couldn’t move to either thus mailing list or an
> issue or wiki on github?
>
> Regards,
> Dave
>
> > On Apr 1, 2018, at 10:38 PM, Karthik Ramasamy <ka...@streaml.io>
> wrote:
> >
> > Tom - Added a bunch of feedback
> >
> > cheers
> > /karthik
> >
> >> On Sun, Apr 1, 2018 at 3:24 PM, Karthik Ramasamy <ka...@streaml.io>
> wrote:
> >>
> >> Tom - permissions look good now.
> >>
> >> On Sun, Apr 1, 2018 at 3:03 PM Thomas Cooper <to...@gmail.com>
> >> wrote:
> >>
> >>> Hi all,
> >>>
> >>> Hopefully the share permissions are correct on the doc now:
> >>>
> >>> <goog_1394670011>
> >>> https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwt
> >>> i7Q0hmW6lmgyk8/edit?usp=sharing
> >>>
> >>> Look forward to hearing what you think.
> >>>
> >>> Also is it worth making an issue for this?
> >>>
> >>> Cheers,
> >>>
> >>> Tom Cooper
> >>> W: www.tomcooper.org.uk | Twitter: @tomncooper
> >>> <https://twitter.com/tomncooper>
> >>>
> >>>> On 30 March 2018 at 20:05, Karthik Ramasamy <kr...@gmail.com>
> wrote:
> >>>>
> >>>> Thanks Tom. It will be a great addition. Unable to access the doc.
> Could
> >>>> you give permission?
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>>> On Mar 30, 2018, at 6:08 PM, Thomas Cooper <to...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> Hi all,
> >>>>>
> >>>>> After chatting to the Heron Team at Twitter and Microsoft I have
> >>> written
> >>>> up
> >>>>> a proposal for giving Heron the ability to preserve state across a
> >>>> topology
> >>>>> update (change in component parallelism). This is problem for
> >>> components
> >>>>> with incoming fields based connections as keys will not hash to the
> >>> same
> >>>>> task ID after an update. I have defined the problem (as I see it) and
> >>>>> propsoed two solutions in the google doc below:
> >>>>>
> >>>>> https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwt
> >>>> i7Q0hmW6lmgyk8/edit?usp=sharing
> >>>>>
> >>>>> I would love to get feedback on these proposals. Feel free to blow
> >>> holes
> >>>> in
> >>>>> my assumptions and correct any misinterpretations I have made!
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Tom Cooper
> >>>>> W: www.tomcooper.org.uk | Twitter: @tomncooper
> >>>>> <https://twitter.com/tomncooper>
> >>>>
> >>>
> >>
>
>
Re: Stateful Updates
Posted by Dave Fisher <da...@comcast.net>.
Hi -
Any reason this document couldn’t move to either thus mailing list or an issue or wiki on github?
Regards,
Dave
> On Apr 1, 2018, at 10:38 PM, Karthik Ramasamy <ka...@streaml.io> wrote:
>
> Tom - Added a bunch of feedback
>
> cheers
> /karthik
>
>> On Sun, Apr 1, 2018 at 3:24 PM, Karthik Ramasamy <ka...@streaml.io> wrote:
>>
>> Tom - permissions look good now.
>>
>> On Sun, Apr 1, 2018 at 3:03 PM Thomas Cooper <to...@gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> Hopefully the share permissions are correct on the doc now:
>>>
>>> <goog_1394670011>
>>> https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwt
>>> i7Q0hmW6lmgyk8/edit?usp=sharing
>>>
>>> Look forward to hearing what you think.
>>>
>>> Also is it worth making an issue for this?
>>>
>>> Cheers,
>>>
>>> Tom Cooper
>>> W: www.tomcooper.org.uk | Twitter: @tomncooper
>>> <https://twitter.com/tomncooper>
>>>
>>>> On 30 March 2018 at 20:05, Karthik Ramasamy <kr...@gmail.com> wrote:
>>>>
>>>> Thanks Tom. It will be a great addition. Unable to access the doc. Could
>>>> you give permission?
>>>>
>>>> Sent from my iPhone
>>>>
>>>>> On Mar 30, 2018, at 6:08 PM, Thomas Cooper <to...@gmail.com>
>>>> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> After chatting to the Heron Team at Twitter and Microsoft I have
>>> written
>>>> up
>>>>> a proposal for giving Heron the ability to preserve state across a
>>>> topology
>>>>> update (change in component parallelism). This is problem for
>>> components
>>>>> with incoming fields based connections as keys will not hash to the
>>> same
>>>>> task ID after an update. I have defined the problem (as I see it) and
>>>>> propsoed two solutions in the google doc below:
>>>>>
>>>>> https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwt
>>>> i7Q0hmW6lmgyk8/edit?usp=sharing
>>>>>
>>>>> I would love to get feedback on these proposals. Feel free to blow
>>> holes
>>>> in
>>>>> my assumptions and correct any misinterpretations I have made!
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Tom Cooper
>>>>> W: www.tomcooper.org.uk | Twitter: @tomncooper
>>>>> <https://twitter.com/tomncooper>
>>>>
>>>
>>
Re: Stateful Updates
Posted by Karthik Ramasamy <ka...@streaml.io>.
Tom - Added a bunch of feedback
cheers
/karthik
On Sun, Apr 1, 2018 at 3:24 PM, Karthik Ramasamy <ka...@streaml.io> wrote:
> Tom - permissions look good now.
>
> On Sun, Apr 1, 2018 at 3:03 PM Thomas Cooper <to...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> Hopefully the share permissions are correct on the doc now:
>>
>> <goog_1394670011>
>> https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwt
>> i7Q0hmW6lmgyk8/edit?usp=sharing
>>
>> Look forward to hearing what you think.
>>
>> Also is it worth making an issue for this?
>>
>> Cheers,
>>
>> Tom Cooper
>> W: www.tomcooper.org.uk | Twitter: @tomncooper
>> <https://twitter.com/tomncooper>
>>
>> On 30 March 2018 at 20:05, Karthik Ramasamy <kr...@gmail.com> wrote:
>>
>> > Thanks Tom. It will be a great addition. Unable to access the doc. Could
>> > you give permission?
>> >
>> > Sent from my iPhone
>> >
>> > > On Mar 30, 2018, at 6:08 PM, Thomas Cooper <to...@gmail.com>
>> > wrote:
>> > >
>> > > Hi all,
>> > >
>> > > After chatting to the Heron Team at Twitter and Microsoft I have
>> written
>> > up
>> > > a proposal for giving Heron the ability to preserve state across a
>> > topology
>> > > update (change in component parallelism). This is problem for
>> components
>> > > with incoming fields based connections as keys will not hash to the
>> same
>> > > task ID after an update. I have defined the problem (as I see it) and
>> > > propsoed two solutions in the google doc below:
>> > >
>> > > https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwt
>> > i7Q0hmW6lmgyk8/edit?usp=sharing
>> > >
>> > > I would love to get feedback on these proposals. Feel free to blow
>> holes
>> > in
>> > > my assumptions and correct any misinterpretations I have made!
>> > >
>> > > Thanks,
>> > >
>> > > Tom Cooper
>> > > W: www.tomcooper.org.uk | Twitter: @tomncooper
>> > > <https://twitter.com/tomncooper>
>> >
>>
>
Re: Stateful Updates
Posted by Karthik Ramasamy <ka...@streaml.io>.
Tom - permissions look good now.
On Sun, Apr 1, 2018 at 3:03 PM Thomas Cooper <to...@gmail.com> wrote:
> Hi all,
>
> Hopefully the share permissions are correct on the doc now:
>
> <goog_1394670011>
>
> https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwti7Q0hmW6lmgyk8/edit?usp=sharing
>
> Look forward to hearing what you think.
>
> Also is it worth making an issue for this?
>
> Cheers,
>
> Tom Cooper
> W: www.tomcooper.org.uk | Twitter: @tomncooper
> <https://twitter.com/tomncooper>
>
> On 30 March 2018 at 20:05, Karthik Ramasamy <kr...@gmail.com> wrote:
>
> > Thanks Tom. It will be a great addition. Unable to access the doc. Could
> > you give permission?
> >
> > Sent from my iPhone
> >
> > > On Mar 30, 2018, at 6:08 PM, Thomas Cooper <to...@gmail.com>
> > wrote:
> > >
> > > Hi all,
> > >
> > > After chatting to the Heron Team at Twitter and Microsoft I have
> written
> > up
> > > a proposal for giving Heron the ability to preserve state across a
> > topology
> > > update (change in component parallelism). This is problem for
> components
> > > with incoming fields based connections as keys will not hash to the
> same
> > > task ID after an update. I have defined the problem (as I see it) and
> > > propsoed two solutions in the google doc below:
> > >
> > > https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwt
> > i7Q0hmW6lmgyk8/edit?usp=sharing
> > >
> > > I would love to get feedback on these proposals. Feel free to blow
> holes
> > in
> > > my assumptions and correct any misinterpretations I have made!
> > >
> > > Thanks,
> > >
> > > Tom Cooper
> > > W: www.tomcooper.org.uk | Twitter: @tomncooper
> > > <https://twitter.com/tomncooper>
> >
>