You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Hiroyuki Adachi <ha...@yahoo-corp.jp> on 2017/01/17 02:46:57 UTC
Question about adding our script at pre/post start and pre/post stop
actions
Hello,
We want to add our script at pre/post start and pre/post stop actions.
When we upgrade our production environment,
we need to check state of NameNode and wait for completion of HiveServer2 connections, and so on.
I know that we can do that with implementing custom services,
but we do not willing to folk the code for such small jobs.
Thanks,
Hiroyuki Adachi
Re: Question about adding our script at pre/post start and pre/post
stop actions
Posted by Hiroyuki Adachi <ha...@yahoo-corp.jp>.
Thank you for your kind explanation. I’ll try it!
Thanks,
Hiroyuki Adachi
--
I see, you'll need to take a look at the pre-* hooks in the stack, e.g.,
pre-install, pre-start.
E.g.,
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START
You may potentially need to modify script.py so that calling a start
method is allowed to call a pre_start method.
You also need to figure out if your pre_start method should be called
during an RU/EU, and if it happens before/after the pre_upgrade_restart
method that today is called between the stop and start during a restart
during RU/EU.
Thanks,
Alejandro
On 1/18/17, 6:53 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp> wrote:
>Hi Alejandro,
>
>I want to customize start/stop/restart actions not only during upgrade
>operation.
>
>In my understanding, a component script has start and stop methods,
>and they will be called when I push "Start/Stop" buttons of a component
>on Ambari UI.
>I want to call pre-start and post-start methods in my script
>before and after calling provided start method.
>
>On NAMENODE component, for example,
># current action flow
>1. push "Start" button on Ambari UI
>2. start method in namenode.py will be called
>https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/
>resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
>
># I want to change the action flow to below
>1. push "Start" button on Ambari UI
>2. pre-start method in myhook-namenode.py will be called
>3. start method in namenode.py will be called
>4. post-start method in myhook-namenode.py will be called
>
>Thanks,
>Hiroyuki Adachi
>
>--
> Adachisan,
>
>
> The usual operations of "restart" also appear in the upgrade packs.
>E.g.,
>
>
>https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resourc
>e
> s/stacks/HDP/2.4/upgrades/upgrade-2.5.xml
>
>
> <processing>
> <service name="ZOOKEEPER">
> <component name="ZOOKEEPER_SERVER">
> <pre-upgrade>
> <task xsi:type="configure" id="zookeeper_log4j_parameterize"
>/>
> </pre-upgrade>
> <pre-downgrade/>
> <upgrade>
> <task xsi:type="restart-task" /> <!-- Exact action to invoke
> during the upgrade portion. -->
> </upgrade>
> </component>
>
>
>
> Thanks,
> Alejandro
>
> On 1/17/17, 10:01 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp> wrote:
>
> >Hi Alejandro,
> >
> >Thank you for the response.
> >I'm using HDP, and your information of the upgrade packs is useful
>for me.
> >
> >But, how about usual operations? (e.g. restart NameNode from Ambari)
> >Can I hook pre/post start, stop, and restart actions of components?
> >It is desirable that I can implement such actions in a similar way
>to the
> >upgrade packs.
> >
> >Thanks,
> >Hiroyuki Adachi
> >
> >----
> > Hi Adachisan,
> >
> > What stack are you using?
> > If you're using HDP, you can modify the upgrade packs (xml
>files) with
> > additional steps like calling functions in the python files to
> >perform any
> > additional logic.
> > E.g.,
> >
> > <service name="HDFS">
> >
> > ...
> > <component name="NAMENODE">
> > ...
> > <post-upgrade>
> > <task xsi:type="execute">
> > <summary>Wait for HiveServer2 Connections</summary>
> > <script>scripts/namenode_upgrade.py</script>
> > <function>wait_for_hive</function>
> > </task>
> > </post-upgrade>
> >
> >
> >
> > In namenode_upgrade.py
> >
> > def wait_for_hive(self, env):
> > import params
> > # Ensure that Hive Server is actually present in the cluster
> >
> > Thanks,
> >
> >
> > Alejandro
> >
> > On 1/16/17, 6:46 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp>
>wrote:
> >
> > >Hello,
> > >
> > >We want to add our script at pre/post start and pre/post stop
> >actions.
> > >
> > >When we upgrade our production environment,
> > >we need to check state of NameNode and wait for completion of
> >HiveServer2
> > >connections, and so on.
> > >
> > >I know that we can do that with implementing custom services,
> > >but we do not willing to folk the code for such small jobs.
> > >
> > >Thanks,
> > >Hiroyuki Adachi
> > >
> > >
> >
> >
> >
>
>
>
Re: Question about adding our script at pre/post start and pre/post
stop actions
Posted by Alejandro Fernandez <af...@hortonworks.com>.
I see, you'll need to take a look at the pre-* hooks in the stack, e.g.,
pre-install, pre-start.
E.g.,
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START
You may potentially need to modify script.py so that calling a start
method is allowed to call a pre_start method.
You also need to figure out if your pre_start method should be called
during an RU/EU, and if it happens before/after the pre_upgrade_restart
method that today is called between the stop and start during a restart
during RU/EU.
Thanks,
Alejandro
On 1/18/17, 6:53 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp> wrote:
>Hi Alejandro,
>
>I want to customize start/stop/restart actions not only during upgrade
>operation.
>
>In my understanding, a component script has start and stop methods,
>and they will be called when I push "Start/Stop" buttons of a component
>on Ambari UI.
>I want to call pre-start and post-start methods in my script
>before and after calling provided start method.
>
>On NAMENODE component, for example,
># current action flow
>1. push "Start" button on Ambari UI
>2. start method in namenode.py will be called
>https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/
>resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
>
># I want to change the action flow to below
>1. push "Start" button on Ambari UI
>2. pre-start method in myhook-namenode.py will be called
>3. start method in namenode.py will be called
>4. post-start method in myhook-namenode.py will be called
>
>Thanks,
>Hiroyuki Adachi
>
>--
> Adachisan,
>
>
> The usual operations of "restart" also appear in the upgrade packs.
>E.g.,
>
>
>https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resourc
>e
> s/stacks/HDP/2.4/upgrades/upgrade-2.5.xml
>
>
> <processing>
> <service name="ZOOKEEPER">
> <component name="ZOOKEEPER_SERVER">
> <pre-upgrade>
> <task xsi:type="configure" id="zookeeper_log4j_parameterize"
>/>
> </pre-upgrade>
> <pre-downgrade/>
> <upgrade>
> <task xsi:type="restart-task" /> <!-- Exact action to invoke
> during the upgrade portion. -->
> </upgrade>
> </component>
>
>
>
> Thanks,
> Alejandro
>
> On 1/17/17, 10:01 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp> wrote:
>
> >Hi Alejandro,
> >
> >Thank you for the response.
> >I'm using HDP, and your information of the upgrade packs is useful
>for me.
> >
> >But, how about usual operations? (e.g. restart NameNode from Ambari)
> >Can I hook pre/post start, stop, and restart actions of components?
> >It is desirable that I can implement such actions in a similar way
>to the
> >upgrade packs.
> >
> >Thanks,
> >Hiroyuki Adachi
> >
> >----
> > Hi Adachisan,
> >
> > What stack are you using?
> > If you're using HDP, you can modify the upgrade packs (xml
>files) with
> > additional steps like calling functions in the python files to
> >perform any
> > additional logic.
> > E.g.,
> >
> > <service name="HDFS">
> >
> > ...
> > <component name="NAMENODE">
> > ...
> > <post-upgrade>
> > <task xsi:type="execute">
> > <summary>Wait for HiveServer2 Connections</summary>
> > <script>scripts/namenode_upgrade.py</script>
> > <function>wait_for_hive</function>
> > </task>
> > </post-upgrade>
> >
> >
> >
> > In namenode_upgrade.py
> >
> > def wait_for_hive(self, env):
> > import params
> > # Ensure that Hive Server is actually present in the cluster
> >
> > Thanks,
> >
> >
> > Alejandro
> >
> > On 1/16/17, 6:46 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp>
>wrote:
> >
> > >Hello,
> > >
> > >We want to add our script at pre/post start and pre/post stop
> >actions.
> > >
> > >When we upgrade our production environment,
> > >we need to check state of NameNode and wait for completion of
> >HiveServer2
> > >connections, and so on.
> > >
> > >I know that we can do that with implementing custom services,
> > >but we do not willing to folk the code for such small jobs.
> > >
> > >Thanks,
> > >Hiroyuki Adachi
> > >
> > >
> >
> >
> >
>
>
>
Re: Question about adding our script at pre/post start and pre/post
stop actions
Posted by Hiroyuki Adachi <ha...@yahoo-corp.jp>.
Hi Alejandro,
I want to customize start/stop/restart actions not only during upgrade operation.
In my understanding, a component script has start and stop methods,
and they will be called when I push "Start/Stop" buttons of a component on Ambari UI.
I want to call pre-start and post-start methods in my script
before and after calling provided start method.
On NAMENODE component, for example,
# current action flow
1. push "Start" button on Ambari UI
2. start method in namenode.py will be called
https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/
resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
# I want to change the action flow to below
1. push "Start" button on Ambari UI
2. pre-start method in myhook-namenode.py will be called
3. start method in namenode.py will be called
4. post-start method in myhook-namenode.py will be called
Thanks,
Hiroyuki Adachi
--
Adachisan,
The usual operations of "restart" also appear in the upgrade packs. E.g.,
https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resource
s/stacks/HDP/2.4/upgrades/upgrade-2.5.xml
<processing>
<service name="ZOOKEEPER">
<component name="ZOOKEEPER_SERVER">
<pre-upgrade>
<task xsi:type="configure" id="zookeeper_log4j_parameterize" />
</pre-upgrade>
<pre-downgrade/>
<upgrade>
<task xsi:type="restart-task" /> <!-- Exact action to invoke
during the upgrade portion. -->
</upgrade>
</component>
Thanks,
Alejandro
On 1/17/17, 10:01 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp> wrote:
>Hi Alejandro,
>
>Thank you for the response.
>I'm using HDP, and your information of the upgrade packs is useful for me.
>
>But, how about usual operations? (e.g. restart NameNode from Ambari)
>Can I hook pre/post start, stop, and restart actions of components?
>It is desirable that I can implement such actions in a similar way to the
>upgrade packs.
>
>Thanks,
>Hiroyuki Adachi
>
>----
> Hi Adachisan,
>
> What stack are you using?
> If you're using HDP, you can modify the upgrade packs (xml files) with
> additional steps like calling functions in the python files to
>perform any
> additional logic.
> E.g.,
>
> <service name="HDFS">
>
> ...
> <component name="NAMENODE">
> ...
> <post-upgrade>
> <task xsi:type="execute">
> <summary>Wait for HiveServer2 Connections</summary>
> <script>scripts/namenode_upgrade.py</script>
> <function>wait_for_hive</function>
> </task>
> </post-upgrade>
>
>
>
> In namenode_upgrade.py
>
> def wait_for_hive(self, env):
> import params
> # Ensure that Hive Server is actually present in the cluster
>
> Thanks,
>
>
> Alejandro
>
> On 1/16/17, 6:46 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp> wrote:
>
> >Hello,
> >
> >We want to add our script at pre/post start and pre/post stop
>actions.
> >
> >When we upgrade our production environment,
> >we need to check state of NameNode and wait for completion of
>HiveServer2
> >connections, and so on.
> >
> >I know that we can do that with implementing custom services,
> >but we do not willing to folk the code for such small jobs.
> >
> >Thanks,
> >Hiroyuki Adachi
> >
> >
>
>
>
Re: Question about adding our script at pre/post start and pre/post
stop actions
Posted by Alejandro Fernandez <af...@hortonworks.com>.
Adachisan,
The usual operations of "restart" also appear in the upgrade packs. E.g.,
https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resource
s/stacks/HDP/2.4/upgrades/upgrade-2.5.xml
<processing>
<service name="ZOOKEEPER">
<component name="ZOOKEEPER_SERVER">
<pre-upgrade>
<task xsi:type="configure" id="zookeeper_log4j_parameterize" />
</pre-upgrade>
<pre-downgrade/>
<upgrade>
<task xsi:type="restart-task" /> <!-- Exact action to invoke
during the upgrade portion. -->
</upgrade>
</component>
Thanks,
Alejandro
On 1/17/17, 10:01 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp> wrote:
>Hi Alejandro,
>
>Thank you for the response.
>I'm using HDP, and your information of the upgrade packs is useful for me.
>
>But, how about usual operations? (e.g. restart NameNode from Ambari)
>Can I hook pre/post start, stop, and restart actions of components?
>It is desirable that I can implement such actions in a similar way to the
>upgrade packs.
>
>Thanks,
>Hiroyuki Adachi
>
>----
> Hi Adachisan,
>
> What stack are you using?
> If you're using HDP, you can modify the upgrade packs (xml files) with
> additional steps like calling functions in the python files to
>perform any
> additional logic.
> E.g.,
>
> <service name="HDFS">
>
> ...
> <component name="NAMENODE">
> ...
> <post-upgrade>
> <task xsi:type="execute">
> <summary>Wait for HiveServer2 Connections</summary>
> <script>scripts/namenode_upgrade.py</script>
> <function>wait_for_hive</function>
> </task>
> </post-upgrade>
>
>
>
> In namenode_upgrade.py
>
> def wait_for_hive(self, env):
> import params
> # Ensure that Hive Server is actually present in the cluster
>
> Thanks,
>
>
> Alejandro
>
> On 1/16/17, 6:46 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp> wrote:
>
> >Hello,
> >
> >We want to add our script at pre/post start and pre/post stop
>actions.
> >
> >When we upgrade our production environment,
> >we need to check state of NameNode and wait for completion of
>HiveServer2
> >connections, and so on.
> >
> >I know that we can do that with implementing custom services,
> >but we do not willing to folk the code for such small jobs.
> >
> >Thanks,
> >Hiroyuki Adachi
> >
> >
>
>
>
Re: Question about adding our script at pre/post start and pre/post
stop actions
Posted by Hiroyuki Adachi <ha...@yahoo-corp.jp>.
Hi Alejandro,
Thank you for the response.
I'm using HDP, and your information of the upgrade packs is useful for me.
But, how about usual operations? (e.g. restart NameNode from Ambari)
Can I hook pre/post start, stop, and restart actions of components?
It is desirable that I can implement such actions in a similar way to the upgrade packs.
Thanks,
Hiroyuki Adachi
----
Hi Adachisan,
What stack are you using?
If you're using HDP, you can modify the upgrade packs (xml files) with
additional steps like calling functions in the python files to perform any
additional logic.
E.g.,
<service name="HDFS">
...
<component name="NAMENODE">
...
<post-upgrade>
<task xsi:type="execute">
<summary>Wait for HiveServer2 Connections</summary>
<script>scripts/namenode_upgrade.py</script>
<function>wait_for_hive</function>
</task>
</post-upgrade>
In namenode_upgrade.py
def wait_for_hive(self, env):
import params
# Ensure that Hive Server is actually present in the cluster
Thanks,
Alejandro
On 1/16/17, 6:46 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp> wrote:
>Hello,
>
>We want to add our script at pre/post start and pre/post stop actions.
>
>When we upgrade our production environment,
>we need to check state of NameNode and wait for completion of HiveServer2
>connections, and so on.
>
>I know that we can do that with implementing custom services,
>but we do not willing to folk the code for such small jobs.
>
>Thanks,
>Hiroyuki Adachi
>
>
Re: Question about adding our script at pre/post start and pre/post
stop actions
Posted by Alejandro Fernandez <af...@hortonworks.com>.
Hi Adachisan,
What stack are you using?
If you're using HDP, you can modify the upgrade packs (xml files) with
additional steps like calling functions in the python files to perform any
additional logic.
E.g.,
<service name="HDFS">
...
<component name="NAMENODE">
...
<post-upgrade>
<task xsi:type="execute">
<summary>Wait for HiveServer2 Connections</summary>
<script>scripts/namenode_upgrade.py</script>
<function>wait_for_hive</function>
</task>
</post-upgrade>
In namenode_upgrade.py
def wait_for_hive(self, env):
import params
# Ensure that Hive Server is actually present in the cluster
Thanks,
Alejandro
On 1/16/17, 6:46 PM, "Hiroyuki Adachi" <ha...@yahoo-corp.jp> wrote:
>Hello,
>
>We want to add our script at pre/post start and pre/post stop actions.
>
>When we upgrade our production environment,
>we need to check state of NameNode and wait for completion of HiveServer2
>connections, and so on.
>
>I know that we can do that with implementing custom services,
>but we do not willing to folk the code for such small jobs.
>
>Thanks,
>Hiroyuki Adachi
>
>