You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by pradeep kumbhar <pr...@gmail.com> on 2016/03/29 12:35:15 UTC
Why does `-originalAppId` option for dtcli launch command require app-package-file-path
Hi,
I was using dtcli to restart a killed application and observed that we need
to provide application package file path as:
dt> *launch -originalAppId application_1456301553414_0138
~/pi-demo-3.1.1.apa*
As per my knowledge, while doing a restart of killed app, we copy the
contents of killed application directory to new application directory and
then do the restart.
The question I have is, if everything needed for app restart is available
under killed application directory, why do we need to provide app package
file path?
--
*Regards,*
*-pradeep*
Re: Why does `-originalAppId` option for dtcli launch command require app-package-file-path
Posted by pradeep kumbhar <pr...@gmail.com>.
Thanks for the clarification David.
On Fri, Apr 1, 2016 at 1:56 AM, David Yan <da...@datatorrent.com> wrote:
> Pradeep,
>
> I don't think you can re-use an old app id in YARN, and I don't think it's
> a good idea even if it's possible, since the app id is supposedly globally
> unique to identify an application instance.
>
> David
>
> On Tue, Mar 29, 2016 at 10:25 PM, pradeep kumbhar <
> pradeep.n.kumbhar@gmail.com> wrote:
>
> > David,
> > I am not sure but can we restart the app in same directory as that of
> > killed app directory. That way copying of whole data for killed app
> (which
> > could be huge in some cases) will be avoided.
> > I know that currently when we restart the app, a new app_id is assigned
> to
> > it. Are there any complications in using the same app id and the
> directory?
> >
> > On Wed, Mar 30, 2016 at 10:51 AM, Thomas Weise <th...@datatorrent.com>
> > wrote:
> >
> > > Ashwin,
> > >
> > > This is for restarting the application from previous state
> (configuration
> > > was already applied).
> > >
> > > You can change properties on the running app.
> > >
> > > Thomas
> > >
> > > On Tue, Mar 29, 2016 at 11:51 AM, Ashwin Chandra Putta <
> > > ashwinchandrap@gmail.com> wrote:
> > >
> > > > +1 for this feature, this is help reduce the steps to relaunch an
> > > > application. If we do not have any changes to app package /
> properties,
> > > we
> > > > should be able to launch without providing app package/properties and
> > > > automatically picking the previously supplied properties file and
> > > > additional properties.
> > > >
> > > > Also, -conf, -apconf, -D <property>=<value> should be supported even
> > when
> > > > using the existing app package from HDFS.
> > > >
> > > > Regards,
> > > > Ashwin.
> > > >
> > > > On Tue, Mar 29, 2016 at 11:30 AM, David Yan <da...@datatorrent.com>
> > > wrote:
> > > >
> > > > > The ability to restart an app without app package will require some
> > > > changes
> > > > > in Apex Core.
> > > > >
> > > > > I created a ticket to track this issue:
> > > > >
> > > > > https://issues.apache.org/jira/browse/APEXCORE-411
> > > > >
> > > > > David
> > > > >
> > > > > On Tue, Mar 29, 2016 at 6:12 AM, Thomas Weise <
> > thomas@datatorrent.com>
> > > > > wrote:
> > > > >
> > > > > > It is possible to relaunch an application with new jar files.
> > > However,
> > > > we
> > > > > > should also have an option to restart with those already in HDFS.
> > > > > >
> > > > > > --
> > > > > > sent from mobile
> > > > > > On Mar 29, 2016 3:35 AM, "pradeep kumbhar" <
> > > > pradeep.n.kumbhar@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I was using dtcli to restart a killed application and observed
> > that
> > > > we
> > > > > > need
> > > > > > > to provide application package file path as:
> > > > > > >
> > > > > > > dt> *launch -originalAppId application_1456301553414_0138
> > > > > > > ~/pi-demo-3.1.1.apa*
> > > > > > >
> > > > > > >
> > > > > > > As per my knowledge, while doing a restart of killed app, we
> copy
> > > the
> > > > > > > contents of killed application directory to new application
> > > directory
> > > > > and
> > > > > > > then do the restart.
> > > > > > >
> > > > > > > The question I have is, if everything needed for app restart is
> > > > > available
> > > > > > > under killed application directory, why do we need to provide
> app
> > > > > package
> > > > > > > file path?
> > > > > > >
> > > > > > > --
> > > > > > > *Regards,*
> > > > > > > *-pradeep*
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Regards,
> > > > Ashwin.
> > > >
> > >
> >
> >
> >
> > --
> >
> > *Regards,*
> > *-pradeep*
> >
>
--
*Regards,*
*-pradeep*
Re: Why does `-originalAppId` option for dtcli launch command require app-package-file-path
Posted by David Yan <da...@datatorrent.com>.
Pradeep,
I don't think you can re-use an old app id in YARN, and I don't think it's
a good idea even if it's possible, since the app id is supposedly globally
unique to identify an application instance.
David
On Tue, Mar 29, 2016 at 10:25 PM, pradeep kumbhar <
pradeep.n.kumbhar@gmail.com> wrote:
> David,
> I am not sure but can we restart the app in same directory as that of
> killed app directory. That way copying of whole data for killed app (which
> could be huge in some cases) will be avoided.
> I know that currently when we restart the app, a new app_id is assigned to
> it. Are there any complications in using the same app id and the directory?
>
> On Wed, Mar 30, 2016 at 10:51 AM, Thomas Weise <th...@datatorrent.com>
> wrote:
>
> > Ashwin,
> >
> > This is for restarting the application from previous state (configuration
> > was already applied).
> >
> > You can change properties on the running app.
> >
> > Thomas
> >
> > On Tue, Mar 29, 2016 at 11:51 AM, Ashwin Chandra Putta <
> > ashwinchandrap@gmail.com> wrote:
> >
> > > +1 for this feature, this is help reduce the steps to relaunch an
> > > application. If we do not have any changes to app package / properties,
> > we
> > > should be able to launch without providing app package/properties and
> > > automatically picking the previously supplied properties file and
> > > additional properties.
> > >
> > > Also, -conf, -apconf, -D <property>=<value> should be supported even
> when
> > > using the existing app package from HDFS.
> > >
> > > Regards,
> > > Ashwin.
> > >
> > > On Tue, Mar 29, 2016 at 11:30 AM, David Yan <da...@datatorrent.com>
> > wrote:
> > >
> > > > The ability to restart an app without app package will require some
> > > changes
> > > > in Apex Core.
> > > >
> > > > I created a ticket to track this issue:
> > > >
> > > > https://issues.apache.org/jira/browse/APEXCORE-411
> > > >
> > > > David
> > > >
> > > > On Tue, Mar 29, 2016 at 6:12 AM, Thomas Weise <
> thomas@datatorrent.com>
> > > > wrote:
> > > >
> > > > > It is possible to relaunch an application with new jar files.
> > However,
> > > we
> > > > > should also have an option to restart with those already in HDFS.
> > > > >
> > > > > --
> > > > > sent from mobile
> > > > > On Mar 29, 2016 3:35 AM, "pradeep kumbhar" <
> > > pradeep.n.kumbhar@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I was using dtcli to restart a killed application and observed
> that
> > > we
> > > > > need
> > > > > > to provide application package file path as:
> > > > > >
> > > > > > dt> *launch -originalAppId application_1456301553414_0138
> > > > > > ~/pi-demo-3.1.1.apa*
> > > > > >
> > > > > >
> > > > > > As per my knowledge, while doing a restart of killed app, we copy
> > the
> > > > > > contents of killed application directory to new application
> > directory
> > > > and
> > > > > > then do the restart.
> > > > > >
> > > > > > The question I have is, if everything needed for app restart is
> > > > available
> > > > > > under killed application directory, why do we need to provide app
> > > > package
> > > > > > file path?
> > > > > >
> > > > > > --
> > > > > > *Regards,*
> > > > > > *-pradeep*
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > Regards,
> > > Ashwin.
> > >
> >
>
>
>
> --
>
> *Regards,*
> *-pradeep*
>
Re: Why does `-originalAppId` option for dtcli launch command require app-package-file-path
Posted by Sandeep Deshmukh <sa...@datatorrent.com>.
We had thought on using same app id and directory in one of our earlier use
cases. The primary consideration was not to play with state of the app that
is already shutdown. If the user wants to restart the same app twice from
the same state (for whatever reason), it will be not be possible if app id
and directory are reused.
Re: Why does `-originalAppId` option for dtcli launch command require app-package-file-path
Posted by pradeep kumbhar <pr...@gmail.com>.
David,
I am not sure but can we restart the app in same directory as that of
killed app directory. That way copying of whole data for killed app (which
could be huge in some cases) will be avoided.
I know that currently when we restart the app, a new app_id is assigned to
it. Are there any complications in using the same app id and the directory?
On Wed, Mar 30, 2016 at 10:51 AM, Thomas Weise <th...@datatorrent.com>
wrote:
> Ashwin,
>
> This is for restarting the application from previous state (configuration
> was already applied).
>
> You can change properties on the running app.
>
> Thomas
>
> On Tue, Mar 29, 2016 at 11:51 AM, Ashwin Chandra Putta <
> ashwinchandrap@gmail.com> wrote:
>
> > +1 for this feature, this is help reduce the steps to relaunch an
> > application. If we do not have any changes to app package / properties,
> we
> > should be able to launch without providing app package/properties and
> > automatically picking the previously supplied properties file and
> > additional properties.
> >
> > Also, -conf, -apconf, -D <property>=<value> should be supported even when
> > using the existing app package from HDFS.
> >
> > Regards,
> > Ashwin.
> >
> > On Tue, Mar 29, 2016 at 11:30 AM, David Yan <da...@datatorrent.com>
> wrote:
> >
> > > The ability to restart an app without app package will require some
> > changes
> > > in Apex Core.
> > >
> > > I created a ticket to track this issue:
> > >
> > > https://issues.apache.org/jira/browse/APEXCORE-411
> > >
> > > David
> > >
> > > On Tue, Mar 29, 2016 at 6:12 AM, Thomas Weise <th...@datatorrent.com>
> > > wrote:
> > >
> > > > It is possible to relaunch an application with new jar files.
> However,
> > we
> > > > should also have an option to restart with those already in HDFS.
> > > >
> > > > --
> > > > sent from mobile
> > > > On Mar 29, 2016 3:35 AM, "pradeep kumbhar" <
> > pradeep.n.kumbhar@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I was using dtcli to restart a killed application and observed that
> > we
> > > > need
> > > > > to provide application package file path as:
> > > > >
> > > > > dt> *launch -originalAppId application_1456301553414_0138
> > > > > ~/pi-demo-3.1.1.apa*
> > > > >
> > > > >
> > > > > As per my knowledge, while doing a restart of killed app, we copy
> the
> > > > > contents of killed application directory to new application
> directory
> > > and
> > > > > then do the restart.
> > > > >
> > > > > The question I have is, if everything needed for app restart is
> > > available
> > > > > under killed application directory, why do we need to provide app
> > > package
> > > > > file path?
> > > > >
> > > > > --
> > > > > *Regards,*
> > > > > *-pradeep*
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> >
> > Regards,
> > Ashwin.
> >
>
--
*Regards,*
*-pradeep*
Re: Why does `-originalAppId` option for dtcli launch command require app-package-file-path
Posted by Ashwin Chandra Putta <as...@gmail.com>.
Thomas, It will be good to have feature to override properties at relaunch
time.
Regards,
Ashwin.
On Mar 29, 2016 10:21 PM, "Thomas Weise" <th...@datatorrent.com> wrote:
> Ashwin,
>
> This is for restarting the application from previous state (configuration
> was already applied).
>
> You can change properties on the running app.
>
> Thomas
>
> On Tue, Mar 29, 2016 at 11:51 AM, Ashwin Chandra Putta <
> ashwinchandrap@gmail.com> wrote:
>
> > +1 for this feature, this is help reduce the steps to relaunch an
> > application. If we do not have any changes to app package / properties,
> we
> > should be able to launch without providing app package/properties and
> > automatically picking the previously supplied properties file and
> > additional properties.
> >
> > Also, -conf, -apconf, -D <property>=<value> should be supported even when
> > using the existing app package from HDFS.
> >
> > Regards,
> > Ashwin.
> >
> > On Tue, Mar 29, 2016 at 11:30 AM, David Yan <da...@datatorrent.com>
> wrote:
> >
> > > The ability to restart an app without app package will require some
> > changes
> > > in Apex Core.
> > >
> > > I created a ticket to track this issue:
> > >
> > > https://issues.apache.org/jira/browse/APEXCORE-411
> > >
> > > David
> > >
> > > On Tue, Mar 29, 2016 at 6:12 AM, Thomas Weise <th...@datatorrent.com>
> > > wrote:
> > >
> > > > It is possible to relaunch an application with new jar files.
> However,
> > we
> > > > should also have an option to restart with those already in HDFS.
> > > >
> > > > --
> > > > sent from mobile
> > > > On Mar 29, 2016 3:35 AM, "pradeep kumbhar" <
> > pradeep.n.kumbhar@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I was using dtcli to restart a killed application and observed that
> > we
> > > > need
> > > > > to provide application package file path as:
> > > > >
> > > > > dt> *launch -originalAppId application_1456301553414_0138
> > > > > ~/pi-demo-3.1.1.apa*
> > > > >
> > > > >
> > > > > As per my knowledge, while doing a restart of killed app, we copy
> the
> > > > > contents of killed application directory to new application
> directory
> > > and
> > > > > then do the restart.
> > > > >
> > > > > The question I have is, if everything needed for app restart is
> > > available
> > > > > under killed application directory, why do we need to provide app
> > > package
> > > > > file path?
> > > > >
> > > > > --
> > > > > *Regards,*
> > > > > *-pradeep*
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> >
> > Regards,
> > Ashwin.
> >
>
Re: Why does `-originalAppId` option for dtcli launch command require app-package-file-path
Posted by Thomas Weise <th...@datatorrent.com>.
Ashwin,
This is for restarting the application from previous state (configuration
was already applied).
You can change properties on the running app.
Thomas
On Tue, Mar 29, 2016 at 11:51 AM, Ashwin Chandra Putta <
ashwinchandrap@gmail.com> wrote:
> +1 for this feature, this is help reduce the steps to relaunch an
> application. If we do not have any changes to app package / properties, we
> should be able to launch without providing app package/properties and
> automatically picking the previously supplied properties file and
> additional properties.
>
> Also, -conf, -apconf, -D <property>=<value> should be supported even when
> using the existing app package from HDFS.
>
> Regards,
> Ashwin.
>
> On Tue, Mar 29, 2016 at 11:30 AM, David Yan <da...@datatorrent.com> wrote:
>
> > The ability to restart an app without app package will require some
> changes
> > in Apex Core.
> >
> > I created a ticket to track this issue:
> >
> > https://issues.apache.org/jira/browse/APEXCORE-411
> >
> > David
> >
> > On Tue, Mar 29, 2016 at 6:12 AM, Thomas Weise <th...@datatorrent.com>
> > wrote:
> >
> > > It is possible to relaunch an application with new jar files. However,
> we
> > > should also have an option to restart with those already in HDFS.
> > >
> > > --
> > > sent from mobile
> > > On Mar 29, 2016 3:35 AM, "pradeep kumbhar" <
> pradeep.n.kumbhar@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I was using dtcli to restart a killed application and observed that
> we
> > > need
> > > > to provide application package file path as:
> > > >
> > > > dt> *launch -originalAppId application_1456301553414_0138
> > > > ~/pi-demo-3.1.1.apa*
> > > >
> > > >
> > > > As per my knowledge, while doing a restart of killed app, we copy the
> > > > contents of killed application directory to new application directory
> > and
> > > > then do the restart.
> > > >
> > > > The question I have is, if everything needed for app restart is
> > available
> > > > under killed application directory, why do we need to provide app
> > package
> > > > file path?
> > > >
> > > > --
> > > > *Regards,*
> > > > *-pradeep*
> > > >
> > >
> >
>
>
>
> --
>
> Regards,
> Ashwin.
>
Re: Why does `-originalAppId` option for dtcli launch command require app-package-file-path
Posted by Pradeep Kumbhar <pr...@datatorrent.com>.
Thanks David.
+1 for adding the feature.
On Wed, Mar 30, 2016 at 12:21 AM, Ashwin Chandra Putta <
ashwinchandrap@gmail.com> wrote:
> +1 for this feature, this is help reduce the steps to relaunch an
> application. If we do not have any changes to app package / properties, we
> should be able to launch without providing app package/properties and
> automatically picking the previously supplied properties file and
> additional properties.
>
> Also, -conf, -apconf, -D <property>=<value> should be supported even when
> using the existing app package from HDFS.
>
> Regards,
> Ashwin.
>
> On Tue, Mar 29, 2016 at 11:30 AM, David Yan <da...@datatorrent.com> wrote:
>
> > The ability to restart an app without app package will require some
> changes
> > in Apex Core.
> >
> > I created a ticket to track this issue:
> >
> > https://issues.apache.org/jira/browse/APEXCORE-411
> >
> > David
> >
> > On Tue, Mar 29, 2016 at 6:12 AM, Thomas Weise <th...@datatorrent.com>
> > wrote:
> >
> > > It is possible to relaunch an application with new jar files. However,
> we
> > > should also have an option to restart with those already in HDFS.
> > >
> > > --
> > > sent from mobile
> > > On Mar 29, 2016 3:35 AM, "pradeep kumbhar" <
> pradeep.n.kumbhar@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I was using dtcli to restart a killed application and observed that
> we
> > > need
> > > > to provide application package file path as:
> > > >
> > > > dt> *launch -originalAppId application_1456301553414_0138
> > > > ~/pi-demo-3.1.1.apa*
> > > >
> > > >
> > > > As per my knowledge, while doing a restart of killed app, we copy the
> > > > contents of killed application directory to new application directory
> > and
> > > > then do the restart.
> > > >
> > > > The question I have is, if everything needed for app restart is
> > available
> > > > under killed application directory, why do we need to provide app
> > package
> > > > file path?
> > > >
> > > > --
> > > > *Regards,*
> > > > *-pradeep*
> > > >
> > >
> >
>
>
>
> --
>
> Regards,
> Ashwin.
>
--
*regards,*
*~pradeep*
Re: Why does `-originalAppId` option for dtcli launch command require app-package-file-path
Posted by Ashwin Chandra Putta <as...@gmail.com>.
+1 for this feature, this is help reduce the steps to relaunch an
application. If we do not have any changes to app package / properties, we
should be able to launch without providing app package/properties and
automatically picking the previously supplied properties file and
additional properties.
Also, -conf, -apconf, -D <property>=<value> should be supported even when
using the existing app package from HDFS.
Regards,
Ashwin.
On Tue, Mar 29, 2016 at 11:30 AM, David Yan <da...@datatorrent.com> wrote:
> The ability to restart an app without app package will require some changes
> in Apex Core.
>
> I created a ticket to track this issue:
>
> https://issues.apache.org/jira/browse/APEXCORE-411
>
> David
>
> On Tue, Mar 29, 2016 at 6:12 AM, Thomas Weise <th...@datatorrent.com>
> wrote:
>
> > It is possible to relaunch an application with new jar files. However, we
> > should also have an option to restart with those already in HDFS.
> >
> > --
> > sent from mobile
> > On Mar 29, 2016 3:35 AM, "pradeep kumbhar" <pr...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I was using dtcli to restart a killed application and observed that we
> > need
> > > to provide application package file path as:
> > >
> > > dt> *launch -originalAppId application_1456301553414_0138
> > > ~/pi-demo-3.1.1.apa*
> > >
> > >
> > > As per my knowledge, while doing a restart of killed app, we copy the
> > > contents of killed application directory to new application directory
> and
> > > then do the restart.
> > >
> > > The question I have is, if everything needed for app restart is
> available
> > > under killed application directory, why do we need to provide app
> package
> > > file path?
> > >
> > > --
> > > *Regards,*
> > > *-pradeep*
> > >
> >
>
--
Regards,
Ashwin.
Re: Why does `-originalAppId` option for dtcli launch command require app-package-file-path
Posted by David Yan <da...@datatorrent.com>.
The ability to restart an app without app package will require some changes
in Apex Core.
I created a ticket to track this issue:
https://issues.apache.org/jira/browse/APEXCORE-411
David
On Tue, Mar 29, 2016 at 6:12 AM, Thomas Weise <th...@datatorrent.com>
wrote:
> It is possible to relaunch an application with new jar files. However, we
> should also have an option to restart with those already in HDFS.
>
> --
> sent from mobile
> On Mar 29, 2016 3:35 AM, "pradeep kumbhar" <pr...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I was using dtcli to restart a killed application and observed that we
> need
> > to provide application package file path as:
> >
> > dt> *launch -originalAppId application_1456301553414_0138
> > ~/pi-demo-3.1.1.apa*
> >
> >
> > As per my knowledge, while doing a restart of killed app, we copy the
> > contents of killed application directory to new application directory and
> > then do the restart.
> >
> > The question I have is, if everything needed for app restart is available
> > under killed application directory, why do we need to provide app package
> > file path?
> >
> > --
> > *Regards,*
> > *-pradeep*
> >
>
Re: Why does `-originalAppId` option for dtcli launch command require app-package-file-path
Posted by Thomas Weise <th...@datatorrent.com>.
It is possible to relaunch an application with new jar files. However, we
should also have an option to restart with those already in HDFS.
--
sent from mobile
On Mar 29, 2016 3:35 AM, "pradeep kumbhar" <pr...@gmail.com>
wrote:
> Hi,
>
> I was using dtcli to restart a killed application and observed that we need
> to provide application package file path as:
>
> dt> *launch -originalAppId application_1456301553414_0138
> ~/pi-demo-3.1.1.apa*
>
>
> As per my knowledge, while doing a restart of killed app, we copy the
> contents of killed application directory to new application directory and
> then do the restart.
>
> The question I have is, if everything needed for app restart is available
> under killed application directory, why do we need to provide app package
> file path?
>
> --
> *Regards,*
> *-pradeep*
>