You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by go...@apache.org on 2015/05/19 07:39:49 UTC
svn commit: r1680175 - in /incubator/slider/site/trunk:
content/docs/slider_specs/application_pkg_upgrade.md templates/nav.html
Author: gourksaha
Date: Tue May 19 05:39:49 2015
New Revision: 1680175
URL: http://svn.apache.org/r1680175
Log:
SLIDER-855 Site documentation for upgrade/reconfig support in Slider
Added:
incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_upgrade.md
Modified:
incubator/slider/site/trunk/templates/nav.html
Added: incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_upgrade.md
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_upgrade.md?rev=1680175&view=auto
==============================================================================
--- incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_upgrade.md (added)
+++ incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_upgrade.md Tue May 19 05:39:49 2015
@@ -0,0 +1,204 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Apache Slider App Packages Rolling Upgrade
+
+##Introduction
+
+From version `0.80.0-incubating` onwards, Slider supports application package rolling upgrade.
+Application package owners can now upgrade to a newer version of their application
+binary and/or configuration, without any downtime.
+
+By saying no downtime, it is primarily the responsibility of the application owners to orchestrate the
+upgrade. It needs to be done in a planned fashion, such that the application appears to be running, as per
+the end-users are concerned. Hence, if **_Apache HBase_** is undergoing an upgrade from version `0.98.4`
+to `1.1.0`, then sufficient **_regionservers_** should be running at all times, which can handle the load
+at the time of the upgrade. Application owners also need to ensure that their application is compatible
+in a mixed version mode. So, while the **_regionservers_** are getting upgraded, the **_master_** will
+still be the older version. Hence `1.1.0` **_regionserver_** should be backward compatible with `0.98.4`
+**_masters_**.
+
+Currently, _Ambari Slider View_ supports automated deployment and management of application packages.
+Automated application package upgrade will be supported soon.
+
+## Phases of Upgrade/Downgrade
+
+#### YARN core (libraries and configurations) upgrade/downgrade
+Running Slider apps will continue to run, with no downtime
+
+#### Slider client upgrade/downgrade
+Does not affect running Slider apps at all. New version of client can co-exist with older versions.
+
+#### Slider Application Master upgrade/downgrade of running applications
+Applications started prior to the start of YARN core upgrade/downgrade, will continue to run with the
+older versions of Slider and Hadoop core libraries. To upgrade running Slider AMs the application needs
+to be stopped and restarted with the new version of the client. There is no support for rolling upgrade
+of Slider AM.
+
+#### Applications deployed by Slider (binaries and configurations) upgrade/downgrade
+This is the what this document is primarily about.
+
+## Upgrade Slider Application Packages
+
+A run-book style list of atomic steps exposed by Slider. These steps will be automated
+by Apache Ambari in a future release. It can be easily orchestrated by a shell script
+or executed manually in the correct order.
+
+### An application owner brings a new package and creates a new instance
+
+Upload a new application package *slider-hbase-app-package_v1.0.zip*
+
+ slider package --install --name MyHBase_Facebook --version 1.0 --package ~/slider-hbase-app-package_v1.0.zip
+
+Create an application *MyHBase_Facebook_Finance*
+
+ slider create MyHBase_Facebook_Finance --template ~/myHBase_template_v1.0.json --resources ~/myHBase_resources_v1.0.json
+
+Application *MyHBase_Facebook_Finance* goes through its usual lifecycle
+
+ slider flex MyHBase_Facebook_Finance --component HBASE_MASTER 5
+
+### The app owner can list all containers of an app instance with filtering options
+
+ slider list MyHBase_Facebook_Finance --containers
+
+ slider list MyHBase_Facebook_Finance --containers --components HBASE_MASTER
+
+ slider list MyHBase_Facebook_Finance --containers --components HBASE_MASTER HBASE_REGIONSERVER --version 1.0
+
+ slider list MyHBase_Facebook_Finance --containers --version 1.0
+
+### At some point, the app owner has a newer version of the package and needs to upgrade
+
+Uploads the newer version 2.0 of the app package
+
+ slider package --install --name MyHBase_Facebook --version 2.0 --package ~/slider-hbase-app-package_v2.0.zip
+
+This command upgrades the internal state of Slider App Master with the new config and resource
+specifications. After this, upgrade command needs to be issued against all containers to upgrade
+them to the new app version e.g. v2.0 in this case. Note, it is recommended to not issue the
+flex command, although Slider does not explicitly block it.
+
+ slider upgrade MyHBase_Facebook_Finance --template ~/myHBase_template_v2.0.json --resources ~/myHBase_resources_v2.0.json
+
+At this point the upgrade orchestrator (Ambari or a script) will rely on the slider `list` command with
+`--containers` option (with additional filter options `--components` and/or `--version`) to find all
+the containers that are running for the application. The orchestrator needs to decide how it wants to
+span/schedule the upgrade commands of every single container. This will primarily depend on the
+type of the application. For e.g. say *HBase* might want to call upgrade on all *Master* containers
+first before calling upgrade on any of the *RegionServer* containers. *Storm* might want to provide
+a 5 min delay between the upgrade of the *Nimbus* and *Supervisor* container(s).
+
+Upgrade all containers with ids `id1, id2, .. idn`. It is possible that all these n containers can
+actually be down at the same time. If this is not desired, then issue n upgrade command with a
+single container id at a time.
+
+ slider upgrade MyHBase_Facebook_Finance --containers id1 id2 .. idn
+
+ e.g. slider upgrade MyHBase_Facebook_Finance --containers container_e03_1427412101162_0001_01_000002 container_e03_1427412101162_0001_01_000003
+
+Upgrade all containers of all roles `role1, role2, .. rolen`. Again, all containers of all the n
+roles can be down at the same time. If this is not desired, use the `--containers` option with
+one container at a time.
+
+ slider upgrade MyHBase_Facebook_Finance --components role1 role2 .. rolen
+
+ e.g. slider upgrade MyHBase_Facebook_Finance --components HBASE_MASTER HBASE_REGIONSERVER
+
+### When do we know that upgrade is complete?
+
+The upgrade command need to be executed against all containers and upgrade can be considered
+complete only if the command `slider list MyHBase_Facebook_Finance --containers --version 1.0`
+returns no containers.
+
+The app owner can also call `slider list MyHBase_Facebook_Finance --containers` to explicitly
+check that all expected containers are running and with the desired new version (e.g. 2.0 in
+this case).
+
+If any issues are encountered in Slider and/or the application during upgrade, the
+application can be downgraded to its original version.
+
+## Downgrade Applications Deployed and Upgraded by Slider
+
+A Slider deployed application might be required to be downgraded to its original version for
+any reason - whether the upgrade was completed in entirety or was in-flight. There are no new
+commands exposed to perform downgrade. It basically involves calling the upgrade command again
+with the older version (e.g. version 1.0 in this case) of the application config and resource
+specifications. It needs to be followed by upgrade commands issued against all containers which
+were upgraded to the newer version.
+
+This command although called upgrade, actually downgrades the internal state of Slider App
+Master with the old v1.0 config and resource specifications
+
+ slider upgrade MyHBase_Facebook_Finance --template ~/myHBase_template_v1.0.json --resources ~/myHBase_resources_v1.0.json
+
+Take the list of all containers returned by the list command -
+
+ slider list MyHBase_Facebook_Finance --containers --version 2.0
+
+And issue upgrade command on all of them (in desired orchestrated fashion) -
+
+ slider upgrade MyHBase_Facebook_Finance --containers id1 id2 .. idn
+
+## Pre and post upgrade hooks
+
+### Pre-upgrade hook (optional)
+The pre-upgrade steps, if provided, will allow applications to execute simple housekeeping tasks
+before Slider actually calls stop operation in an upgrade scenario (specifically if they need to
+be performed in every single container and 1000s of them are running). An example could be to
+send a message to a queue that the current instance of memcached is going down so that the load
+balancer rules can be dynamically updated. Performing long-running tasks or tasks that needs to
+be executed by only 1 instance of a component (n instances of which are running) should be
+performed manually before calling upgrade (or maybe before calling the upgrade command). The
+pre-upgrade hook is not a good candidate for such operations. Additional parameters might have
+to be exposed, like timeout, which can be used to wait at most n seconds (say) after which Slider
+will call the application stop hook even if the pre-upgrade operation is not completed.
+
+### Post-upgrade hook (optional) - not yet supported
+This allows applications to perform simple housekeeping tasks prior to calling start on the new
+version of the application component. This is helpful only if such tasks are required to be
+performed in every single container and 1000s of them are running. This is required only in the
+upgrade scenario and should not be called on new containers created using flex up in non-upgrade
+scenarios. This is tricky, how will Slider determine this? I donât think we need a post-upgrade
+hook? Clarify with Rakesh and remove?
+
+## Upgrade support of simple apps with no packages
+- Upgrade of such simple apps does not involve binaries
+- Upgrade of such simple apps basically means re-configuration only
+- The following can be done to perform the reconfiguration (or upgrade if you prefer to call so)
+ - Download the app package that was internally created by Slider from HDFS (First class Slider
+ command-line support will not be provided for download. So refer to documentation or get
+ Slider/Yarn admin support to find the app package location in HDFS.)
+ - Make necessary changes to metainfo.json and repackage the app
+ - Make necessary updates to appConfig.json and resources.json
+ - Follow the upgrade steps provided in this document to upgrade the simple app
+
+## Assumptions/Recommendations
+
+- Letâs take the scenario when the upgrade command has been called (with new app config and
+ resources) and the orchestrator is in the middle of calling upgrade command on all containers
+ in a certain order/schedule. At this point, it is important to understand that if a container
+ (on which the upgrade command was not called yet) fails, then its replacement container will
+ come up with the newer version of the application.
+- It is recommended to not call `flex` (to increase or decrease the no of containers assigned
+ to a specific role) after the upgrade command has been issued and until upgrade command is
+ issued on all currently running containers (as per current resource specification). Basically
+ avoid calling flex until `slider list --containers --version <old_ver>` returns no more
+ containers. Note, flex command is not explicitly blocked by Slider, hence unexpected
+ behavior of the application might occur if called.
+
+
Modified: incubator/slider/site/trunk/templates/nav.html
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/templates/nav.html?rev=1680175&r1=1680174&r2=1680175&view=diff
==============================================================================
--- incubator/slider/site/trunk/templates/nav.html (original)
+++ incubator/slider/site/trunk/templates/nav.html Tue May 19 05:39:49 2015
@@ -71,6 +71,7 @@
<li id="nav_use_resources"><a href="/docs/configuration/resources.html">Resource Specification</a></li>
<li role="presentation" class="divider"></li>
<li id="nav_use_app_package"><a href="/docs/slider_specs/index.html">Creating App Packages</a></li>
+<li id="nav_use_app_package_upgrade"><a href="/docs/slider_specs/application_pkg_upgrade.html">Upgrading App Packages</a></li>
<li role="presentation" class="divider"></li>
<li id="nav_use_client_config"><a href="/docs/client-configuration.html">Client Configuration</a></li>
<li id="nav_use_exitcodes"><a href="/docs/exitcodes.html">Client Exit Codes</a></li>