You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by Tushar Gosavi <tu...@datatorrent.com> on 2017/01/18 09:41:28 UTC
Re: APEXCORE-294 Clean application shutdown
Dear Community,
I have changed the implementaion plan based on PR comments. Following
is the new proposal for this feature.
https://docs.google.com/a/datatorrent.com/document/d/1hSLH4xi_15OWwW4KY7--LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing
Please provide the feedback. I will update the PR accrodinly.
- Tushar.
On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi <tu...@datatorrent.com> wrote:
> Dear Community,
>
> I have open an pull request for shutting down application by sending
> END_STREAM control tuples from all input operator. This is similar to
> all input operator have stopped after raising ShutdownException.
>
> On receiving shutdown request, master will prepare OPERATOR_STOP
> command for all input partitions, and send it to container as part of
> heartbeat response to container. Container will shutdown the operator
> thread after receiving this command.
>
> https://github.com/apache/apex-core/pull/424
> Please review.
>
> Thanks,
> - Tushar.
Re: APEXCORE-294 Clean application shutdown
Posted by Tushar Gosavi <tu...@datatorrent.com>.
I have updated the documentation for kill v/s shutdown. let me know if
anything more is required.
- Tushar.
On Mon, Jan 23, 2017 at 6:09 PM, Tushar Gosavi <tu...@datatorrent.com> wrote:
> Hi Thomas,
>
> I have updated the pull request as well as design document, will take
> care of apex documentation shortly.
>
> - Tushar.
>
>
> On Fri, Jan 20, 2017 at 8:57 PM, Thomas Weise <th...@apache.org> wrote:
>> As discussed please also look at appropriate user documentation and clarity
>> WRT the CLI commands.
>>
>> Thanks,
>> Thomas
>>
>>
>> On Thu, Jan 19, 2017 at 10:46 PM, Tushar Gosavi <tu...@datatorrent.com>
>> wrote:
>>
>>> Hi Thomas,
>>>
>>> As discussed I will remote the old behaviour of shutdown.
>>>
>>> - Tushar.
>>>
>>> On Fri, Jan 20, 2017 at 10:03 AM, Tushar Gosavi <tu...@datatorrent.com>
>>> wrote:
>>> > Hi Thomas,
>>> >
>>> > I agree with you on the current behavior. The effect of shutdown can
>>> > be achieve with kill. I was more worried about backward compatibility
>>> > issue
>>> > while relaunching the shutdown app, hence kept the default as same as
>>> > before (cli and rest api default). If we don't want to retain the
>>> > relaunch
>>> > behaviour of shutdown app then the code will be more simpler :). Let
>>> > me know, I will update the PR accordinlgy.
>>> >
>>> > - Tushar.
>>> >
>>> >
>>> > On Thu, Jan 19, 2017 at 9:22 PM, Thomas Weise <th...@apache.org> wrote:
>>> >> Tushar,
>>> >>
>>> >> I would like to see the use case for "hard shutdown" vs kill. The
>>> current
>>> >> shutdown behavior isn't useful, confusing and sometimes does not even
>>> end
>>> >> the app (based on feedback received in the past from users). I would
>>> prefer
>>> >> we don't retain it without clear understanding of use case and how it
>>> will
>>> >> really work.
>>> >>
>>> >> Thanks,
>>> >> Thomas
>>> >>
>>> >>
>>> >> On Wed, Jan 18, 2017 at 1:41 AM, Tushar Gosavi <tu...@datatorrent.com>
>>> >> wrote:
>>> >>
>>> >>> Dear Community,
>>> >>>
>>> >>> I have changed the implementaion plan based on PR comments. Following
>>> >>> is the new proposal for this feature.
>>> >>>
>>> >>> https://docs.google.com/a/datatorrent.com/document/d/
>>> 1hSLH4xi_15OWwW4KY7--
>>> >>> LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing
>>> >>>
>>> >>> Please provide the feedback. I will update the PR accrodinly.
>>> >>>
>>> >>> - Tushar.
>>> >>>
>>> >>>
>>> >>> On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi <tushar@datatorrent.com
>>> >
>>> >>> wrote:
>>> >>> > Dear Community,
>>> >>> >
>>> >>> > I have open an pull request for shutting down application by sending
>>> >>> > END_STREAM control tuples from all input operator. This is similar to
>>> >>> > all input operator have stopped after raising ShutdownException.
>>> >>> >
>>> >>> > On receiving shutdown request, master will prepare OPERATOR_STOP
>>> >>> > command for all input partitions, and send it to container as part of
>>> >>> > heartbeat response to container. Container will shutdown the operator
>>> >>> > thread after receiving this command.
>>> >>> >
>>> >>> > https://github.com/apache/apex-core/pull/424
>>> >>> > Please review.
>>> >>> >
>>> >>> > Thanks,
>>> >>> > - Tushar.
>>> >>>
>>>
Re: APEXCORE-294 Clean application shutdown
Posted by Tushar Gosavi <tu...@datatorrent.com>.
Hi Thomas,
I have updated the pull request as well as design document, will take
care of apex documentation shortly.
- Tushar.
On Fri, Jan 20, 2017 at 8:57 PM, Thomas Weise <th...@apache.org> wrote:
> As discussed please also look at appropriate user documentation and clarity
> WRT the CLI commands.
>
> Thanks,
> Thomas
>
>
> On Thu, Jan 19, 2017 at 10:46 PM, Tushar Gosavi <tu...@datatorrent.com>
> wrote:
>
>> Hi Thomas,
>>
>> As discussed I will remote the old behaviour of shutdown.
>>
>> - Tushar.
>>
>> On Fri, Jan 20, 2017 at 10:03 AM, Tushar Gosavi <tu...@datatorrent.com>
>> wrote:
>> > Hi Thomas,
>> >
>> > I agree with you on the current behavior. The effect of shutdown can
>> > be achieve with kill. I was more worried about backward compatibility
>> > issue
>> > while relaunching the shutdown app, hence kept the default as same as
>> > before (cli and rest api default). If we don't want to retain the
>> > relaunch
>> > behaviour of shutdown app then the code will be more simpler :). Let
>> > me know, I will update the PR accordinlgy.
>> >
>> > - Tushar.
>> >
>> >
>> > On Thu, Jan 19, 2017 at 9:22 PM, Thomas Weise <th...@apache.org> wrote:
>> >> Tushar,
>> >>
>> >> I would like to see the use case for "hard shutdown" vs kill. The
>> current
>> >> shutdown behavior isn't useful, confusing and sometimes does not even
>> end
>> >> the app (based on feedback received in the past from users). I would
>> prefer
>> >> we don't retain it without clear understanding of use case and how it
>> will
>> >> really work.
>> >>
>> >> Thanks,
>> >> Thomas
>> >>
>> >>
>> >> On Wed, Jan 18, 2017 at 1:41 AM, Tushar Gosavi <tu...@datatorrent.com>
>> >> wrote:
>> >>
>> >>> Dear Community,
>> >>>
>> >>> I have changed the implementaion plan based on PR comments. Following
>> >>> is the new proposal for this feature.
>> >>>
>> >>> https://docs.google.com/a/datatorrent.com/document/d/
>> 1hSLH4xi_15OWwW4KY7--
>> >>> LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing
>> >>>
>> >>> Please provide the feedback. I will update the PR accrodinly.
>> >>>
>> >>> - Tushar.
>> >>>
>> >>>
>> >>> On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi <tushar@datatorrent.com
>> >
>> >>> wrote:
>> >>> > Dear Community,
>> >>> >
>> >>> > I have open an pull request for shutting down application by sending
>> >>> > END_STREAM control tuples from all input operator. This is similar to
>> >>> > all input operator have stopped after raising ShutdownException.
>> >>> >
>> >>> > On receiving shutdown request, master will prepare OPERATOR_STOP
>> >>> > command for all input partitions, and send it to container as part of
>> >>> > heartbeat response to container. Container will shutdown the operator
>> >>> > thread after receiving this command.
>> >>> >
>> >>> > https://github.com/apache/apex-core/pull/424
>> >>> > Please review.
>> >>> >
>> >>> > Thanks,
>> >>> > - Tushar.
>> >>>
>>
Re: APEXCORE-294 Clean application shutdown
Posted by Thomas Weise <th...@apache.org>.
As discussed please also look at appropriate user documentation and clarity
WRT the CLI commands.
Thanks,
Thomas
On Thu, Jan 19, 2017 at 10:46 PM, Tushar Gosavi <tu...@datatorrent.com>
wrote:
> Hi Thomas,
>
> As discussed I will remote the old behaviour of shutdown.
>
> - Tushar.
>
> On Fri, Jan 20, 2017 at 10:03 AM, Tushar Gosavi <tu...@datatorrent.com>
> wrote:
> > Hi Thomas,
> >
> > I agree with you on the current behavior. The effect of shutdown can
> > be achieve with kill. I was more worried about backward compatibility
> > issue
> > while relaunching the shutdown app, hence kept the default as same as
> > before (cli and rest api default). If we don't want to retain the
> > relaunch
> > behaviour of shutdown app then the code will be more simpler :). Let
> > me know, I will update the PR accordinlgy.
> >
> > - Tushar.
> >
> >
> > On Thu, Jan 19, 2017 at 9:22 PM, Thomas Weise <th...@apache.org> wrote:
> >> Tushar,
> >>
> >> I would like to see the use case for "hard shutdown" vs kill. The
> current
> >> shutdown behavior isn't useful, confusing and sometimes does not even
> end
> >> the app (based on feedback received in the past from users). I would
> prefer
> >> we don't retain it without clear understanding of use case and how it
> will
> >> really work.
> >>
> >> Thanks,
> >> Thomas
> >>
> >>
> >> On Wed, Jan 18, 2017 at 1:41 AM, Tushar Gosavi <tu...@datatorrent.com>
> >> wrote:
> >>
> >>> Dear Community,
> >>>
> >>> I have changed the implementaion plan based on PR comments. Following
> >>> is the new proposal for this feature.
> >>>
> >>> https://docs.google.com/a/datatorrent.com/document/d/
> 1hSLH4xi_15OWwW4KY7--
> >>> LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing
> >>>
> >>> Please provide the feedback. I will update the PR accrodinly.
> >>>
> >>> - Tushar.
> >>>
> >>>
> >>> On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi <tushar@datatorrent.com
> >
> >>> wrote:
> >>> > Dear Community,
> >>> >
> >>> > I have open an pull request for shutting down application by sending
> >>> > END_STREAM control tuples from all input operator. This is similar to
> >>> > all input operator have stopped after raising ShutdownException.
> >>> >
> >>> > On receiving shutdown request, master will prepare OPERATOR_STOP
> >>> > command for all input partitions, and send it to container as part of
> >>> > heartbeat response to container. Container will shutdown the operator
> >>> > thread after receiving this command.
> >>> >
> >>> > https://github.com/apache/apex-core/pull/424
> >>> > Please review.
> >>> >
> >>> > Thanks,
> >>> > - Tushar.
> >>>
>
Re: APEXCORE-294 Clean application shutdown
Posted by Tushar Gosavi <tu...@datatorrent.com>.
Hi Thomas,
As discussed I will remote the old behaviour of shutdown.
- Tushar.
On Fri, Jan 20, 2017 at 10:03 AM, Tushar Gosavi <tu...@datatorrent.com> wrote:
> Hi Thomas,
>
> I agree with you on the current behavior. The effect of shutdown can
> be achieve with kill. I was more worried about backward compatibility
> issue
> while relaunching the shutdown app, hence kept the default as same as
> before (cli and rest api default). If we don't want to retain the
> relaunch
> behaviour of shutdown app then the code will be more simpler :). Let
> me know, I will update the PR accordinlgy.
>
> - Tushar.
>
>
> On Thu, Jan 19, 2017 at 9:22 PM, Thomas Weise <th...@apache.org> wrote:
>> Tushar,
>>
>> I would like to see the use case for "hard shutdown" vs kill. The current
>> shutdown behavior isn't useful, confusing and sometimes does not even end
>> the app (based on feedback received in the past from users). I would prefer
>> we don't retain it without clear understanding of use case and how it will
>> really work.
>>
>> Thanks,
>> Thomas
>>
>>
>> On Wed, Jan 18, 2017 at 1:41 AM, Tushar Gosavi <tu...@datatorrent.com>
>> wrote:
>>
>>> Dear Community,
>>>
>>> I have changed the implementaion plan based on PR comments. Following
>>> is the new proposal for this feature.
>>>
>>> https://docs.google.com/a/datatorrent.com/document/d/1hSLH4xi_15OWwW4KY7--
>>> LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing
>>>
>>> Please provide the feedback. I will update the PR accrodinly.
>>>
>>> - Tushar.
>>>
>>>
>>> On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi <tu...@datatorrent.com>
>>> wrote:
>>> > Dear Community,
>>> >
>>> > I have open an pull request for shutting down application by sending
>>> > END_STREAM control tuples from all input operator. This is similar to
>>> > all input operator have stopped after raising ShutdownException.
>>> >
>>> > On receiving shutdown request, master will prepare OPERATOR_STOP
>>> > command for all input partitions, and send it to container as part of
>>> > heartbeat response to container. Container will shutdown the operator
>>> > thread after receiving this command.
>>> >
>>> > https://github.com/apache/apex-core/pull/424
>>> > Please review.
>>> >
>>> > Thanks,
>>> > - Tushar.
>>>
Re: APEXCORE-294 Clean application shutdown
Posted by Tushar Gosavi <tu...@datatorrent.com>.
Hi Thomas,
I agree with you on the current behavior. The effect of shutdown can
be achieve with kill. I was more worried about backward compatibility
issue
while relaunching the shutdown app, hence kept the default as same as
before (cli and rest api default). If we don't want to retain the
relaunch
behaviour of shutdown app then the code will be more simpler :). Let
me know, I will update the PR accordinlgy.
- Tushar.
On Thu, Jan 19, 2017 at 9:22 PM, Thomas Weise <th...@apache.org> wrote:
> Tushar,
>
> I would like to see the use case for "hard shutdown" vs kill. The current
> shutdown behavior isn't useful, confusing and sometimes does not even end
> the app (based on feedback received in the past from users). I would prefer
> we don't retain it without clear understanding of use case and how it will
> really work.
>
> Thanks,
> Thomas
>
>
> On Wed, Jan 18, 2017 at 1:41 AM, Tushar Gosavi <tu...@datatorrent.com>
> wrote:
>
>> Dear Community,
>>
>> I have changed the implementaion plan based on PR comments. Following
>> is the new proposal for this feature.
>>
>> https://docs.google.com/a/datatorrent.com/document/d/1hSLH4xi_15OWwW4KY7--
>> LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing
>>
>> Please provide the feedback. I will update the PR accrodinly.
>>
>> - Tushar.
>>
>>
>> On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi <tu...@datatorrent.com>
>> wrote:
>> > Dear Community,
>> >
>> > I have open an pull request for shutting down application by sending
>> > END_STREAM control tuples from all input operator. This is similar to
>> > all input operator have stopped after raising ShutdownException.
>> >
>> > On receiving shutdown request, master will prepare OPERATOR_STOP
>> > command for all input partitions, and send it to container as part of
>> > heartbeat response to container. Container will shutdown the operator
>> > thread after receiving this command.
>> >
>> > https://github.com/apache/apex-core/pull/424
>> > Please review.
>> >
>> > Thanks,
>> > - Tushar.
>>
Re: APEXCORE-294 Clean application shutdown
Posted by Thomas Weise <th...@apache.org>.
Tushar,
I would like to see the use case for "hard shutdown" vs kill. The current
shutdown behavior isn't useful, confusing and sometimes does not even end
the app (based on feedback received in the past from users). I would prefer
we don't retain it without clear understanding of use case and how it will
really work.
Thanks,
Thomas
On Wed, Jan 18, 2017 at 1:41 AM, Tushar Gosavi <tu...@datatorrent.com>
wrote:
> Dear Community,
>
> I have changed the implementaion plan based on PR comments. Following
> is the new proposal for this feature.
>
> https://docs.google.com/a/datatorrent.com/document/d/1hSLH4xi_15OWwW4KY7--
> LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing
>
> Please provide the feedback. I will update the PR accrodinly.
>
> - Tushar.
>
>
> On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi <tu...@datatorrent.com>
> wrote:
> > Dear Community,
> >
> > I have open an pull request for shutting down application by sending
> > END_STREAM control tuples from all input operator. This is similar to
> > all input operator have stopped after raising ShutdownException.
> >
> > On receiving shutdown request, master will prepare OPERATOR_STOP
> > command for all input partitions, and send it to container as part of
> > heartbeat response to container. Container will shutdown the operator
> > thread after receiving this command.
> >
> > https://github.com/apache/apex-core/pull/424
> > Please review.
> >
> > Thanks,
> > - Tushar.
>