You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by bu...@apache.org on 2014/07/22 20:17:30 UTC
svn commit: r917125 [4/18] - in /websites/staging/slider/trunk/content: ./
css/ design/ design/registry/ design/specification/ developing/ docs/
docs/configuration/ docs/slider_specs/ downloads/ fonts/ images/ js/
release_notes/
Added: websites/staging/slider/trunk/content/design/registry/p2p_service_registries.html
==============================================================================
--- websites/staging/slider/trunk/content/design/registry/p2p_service_registries.html (added)
+++ websites/staging/slider/trunk/content/design/registry/p2p_service_registries.html Tue Jul 22 18:17:12 2014
@@ -0,0 +1,315 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href="/css/bootstrap.min.css" rel="stylesheet">
+ <link href="/css/bootstrap-theme.min.css" rel="stylesheet">
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
+ <script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <link href="/css/slider.css" rel="stylesheet" type="text/css">
+ <link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
+ <title></title>
+
+</head>
+
+<body>
+<nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+<div class="container-fluid">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-items">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/index.html">Slider</a>
+ </div>
+ <div class="collapse navbar-collapse" id="navbar-items">
+ <ul class="nav navbar-nav">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Project <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_index"><a href="/index.html">Home</a></li>
+<li id="nav_downloads"><a href="/downloads">Downloads</a></li>
+<li id="nav_source"><a href="https://git-wip-us.apache.org/repos/asf/incubator-slider.git"><i class="fa fa-external-link"></i> Source</a></li>
+<li><a href="http://www.apache.org/licenses/LICENSE-2.0"><i class="fa fa-external-link"></i> License</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Community</li>
+<li id="nav_team"><a href="/team.html">Team</a></li>
+<li id="nav_mailinglists"><a href="/mailing_lists.html">Mailing Lists</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Tools</li>
+<li id="nav_jira"><a href="https://issues.apache.org/jira/browse/SLIDER"><i class="fa fa-external-link"></i> Issues</a></li>
+<li id="nav_builds"><a href="https://builds.apache.org/job/Slider-develop"><i class="fa fa-external-link"></i> Builds</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/design">
+ Design <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_design_overview"><a href="/design/architecture.html">Overview</a></li>
+<li id="nav_app_needs"><a href="/docs/slider_specs/application_needs.html">Application Needs</a></li>
+<li id="nav_spec"><a href="/design/specification/index.html">Expected Behavior</a></li>
+<li id="nav_registry"><a href="/design/registry/index.html">Service Registry</a></li>
+<li id="nav_ssl_impl"><a href="/design/ssl_implementation.html">SSL Implementation</a></li>
+<li id="nav_role_history"><a href="/design/rolehistory.html">Role History</a></li>
+</ul>
+</li>
+
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/developing">
+ Developing <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_dev_building"><a href="/developing/building.html">Building</a></li>
+<li id="nav_dev_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_dev_testing"><a href="/developing/testing.html">Testing</a></li>
+<li id="nav_dev_funtesting"><a href="/developing/functional_tests.html">Functional Testing</a></li>
+<li id="nav_dev_manual_testing"><a href="/developing/manual_testing.html">Manual Testing</a></li>
+<li id="nav_releasing"><a href="/developing/releasing.html">Releasing</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Using <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_use_getting_started"><a href="/docs/getting_started.html">Getting Started</a></li>
+<li id="nav_use_manpage"><a href="/docs/manpage.html">Man Page</a></li>
+<li id="nav_use_examples"><a href="/docs/examples.html">Examples</a></li>
+<li id="nav_use_client_config"><a href="/docs/client-configuration.html">Client Configuration</a></li>
+<li id="nav_use_exitcodes"><a href="/docs/exitcodes.html">Client Exit Codes</a></li>
+<li id="nav_use_security"><a href="/docs/security.html">Security</a></li>
+<li id="nav_am_agent_ssl"><a href="/docs/ssl.html">Agent to AM SSL</a></li>
+<li id="nav_use_app_package"><a href="/docs/slider_specs/index.html">Creating App Packages</a></li>
+<li id="nav_use_app_configuration"><a href="/docs/configuration/index.html">App Configuration Model</a></li>
+<li id="nav_use_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_use_troubleshooting"><a href="/docs/troubleshooting.html">Troubleshooting</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ ASF Links <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li><a href="http://www.apache.org"><i class="fa fa-external-link"></i> Apache Software Foundation</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html"><i class="fa fa-external-link"></i> Sponsorship</a></li>
+<li><a href="http://www.apache.org/security/"><i class="fa fa-external-link"></i> Security</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html"><i class="fa fa-external-link"></i> Thanks</a></li>
+</ul>
+</li>
+</ul>
+
+ </div>
+
+</div>
+</nav>
+
+
+<div class="container-fluid">
+<div class="row">
+ <div class="col-md-2" id="sidebar">
+ <div style="text-align: center">
+ <h1><a href="/index.html">Apache Slider (incubating)</a></h1>
+ <hr>
+Latest release: <strong>0.40-incubating</strong><br>
+ <br>
+ <a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
+ </div>
+ <hr>
+ <a href="http://incubator.apache.org"><img id="logo" alt="Apache Incubator ™" class="img-responsive" src="http://incubator.apache.org/images/apache-incubator-logo.png"/></a>
+ </div>
+ <div class="col-md-8 col-md-offset-1">
+
+ <div id="content">
+
+ <h1 class="title"></h1>
+
+ <!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<h1 id="p2p-service-registries-for-apache-slider">P2P Service Registries for Apache Slider</h1>
+<p>Alongside the centralized service registries, there's much prior work on
+P2P discovery systems, especially for mobile and consumer devices.</p>
+<p>They perform some multicast- or distributed hash table-based lookup,
+and tend to have common limitations:</p>
+<ul>
+<li>
+<p>scalability</p>
+</li>
+<li>
+<p>the bootstrapping problem</p>
+</li>
+<li>
+<p>security: can you trust the results to be honest?</p>
+</li>
+<li>
+<p>consistency: can you trust the results to be complete and current?</p>
+</li>
+</ul>
+<p>Bootstrapping is usually done via multicast, possibly then switching
+to unicast for better scale. As multicasting doesn't work in cloud
+infrastructures, none of the services work unmodified in public
+clouds. There's multiple anecdotes of
+<a href="http://aws.amazon.com/simpledb/">Amazon's SimpleDB service</a> being used as a
+registry for in-EC2 applications. At the very least, this service and its
+equivalents in other cloud providers could be used to bootstrap ZK client
+bindings in cloud environments. </p>
+<h2 id="service-location-protocol">Service Location Protocol</h2>
+<p>Service Location Protocol is a protocol for discovery services that came out
+of Sun, Novell and others -it is still available for printer discovery and
+suchlike</p>
+<p>It supports both a multicast discovery mechanism, and a unicast protocol
+to talk to a Directory Agent -an agent that is itself discovered by multicast
+requests, or by listening for the agent's intermittent multicast announcements.</p>
+<p>There's an extension to DHCP, RFC2610, which added the ability for DHCP to
+advertise Directory Agents -this was designed to solve the bootstrap problem
+(though not necessarily security or in-cloud deployment). Apart from a few
+mentions in Windows Server technical notes, it does not appear to exist.</p>
+<ul>
+<li>
+<p>[<a href="http://www.ietf.org/rfc/rfc2608.txt">RFC2608</a>] <em>Service Location Protocol, Version 2</em> , IEEE, 1999</p>
+</li>
+<li>
+<p>[<a href="http://www.ietf.org/rfc/rfc3224.txt">RFC3224</a>] <em>Vendor Extensions for Service Location Protocol, Version 2</em>, IETF, 2003</p>
+</li>
+<li>
+<p>[<a href="http://www.ietf.org/rfc/rfc2610.txt">RFC2610</a>] <em>DHCP Options for Service Location Protocol, IETF, 1999</em></p>
+</li>
+</ul>
+<h2 id="zeroconf"><a href="http://www.zeroconf.org/">Zeroconf</a></h2>
+<p>The multicast discovery service implemented in Apple's Bonjour system
+--multicasting DNS lookups to all peers in the subnet.</p>
+<p>This allows for URLs and hostnames to be dynamically positioned, with
+DNS domain searches allowing for enumeration of service groups. </p>
+<p>This protocol scales very badly; the load on <em>every</em> client in the
+subnet is is O(DNS-queries-across-subnet), hence implicitly <code>O(devices)*O(device-activity)</code>. </p>
+<p>The special domains <code>_tcp.</code>, <code>_udp.</code> and their subdomains can also be
+served up via a normal DNS server.</p>
+<h2 id="jiniapache-river"><a href="http://river.apache.org/doc/specs/html/lookup-spec.html">Jini/Apache River</a></h2>
+<p>Attribute-driven service enumeration, which drives the, Java-client-only
+model of downloading client-side code. There's no requirement for the remote
+services to be in Java, only that drivers are.</p>
+<h2 id="serf"><a href="http://www.serfdom.io/">Serf</a></h2>
+<p>This is a library that implements the <a href="http://www.cs.cornell.edu/~asdas/research/dsn02-swim.pdf">SWIM protocol</a> to propagate information around a cluster. Apparently works in virtualized clusters too. It's already been used in a Flume-on-Hoya provider.</p>
+<h2 id="anubis"><a href="http://sourceforge.net/p/smartfrog/svn/HEAD/tree/trunk/core/components/anubis/">Anubis</a></h2>
+<p>An HP Labs-built <a href="http://sourceforge.net/p/smartfrog/svn/HEAD/tree/trunk/core/components/anubis/doc/HPL-2005-72.pdf?format=raw">High Availability tuple-space</a> in SmartFrog; used in production in some of HP's telco products. An agent publishes facts into the T-Space, and within one heartbeat all other agents have it. One heart-beat later, unless there's been a change in the membership, the publisher knows the others have it. One heartbeat later the agents know the publisher knows it, etc.</p>
+<p>Strengths: </p>
+<ul>
+<li>
+<p>The shared knowledge mechanism permits reasoning and mathematical proofs.</p>
+</li>
+<li>
+<p>Strict ordering between heartbeats implies an ordering in receipt.
+This is stronger than ZK's guarantees.</p>
+</li>
+<li>
+<p>Lets you share a moderate amount of data (the longer the heartbeat
+interval, the more data you can publish).</p>
+</li>
+<li>
+<p>Provided the JVM hosting the Anubis agent is also hosting the service,
+liveness is implicit</p>
+</li>
+<li>
+<p>Secure to the extent that it can be locked down to allow only nodes with
+mutual trust of HTTPS certificates to join the tuple-space.</p>
+</li>
+</ul>
+<p>Weaknesses</p>
+<ul>
+<li>
+<p>(Currently) bootstraps via multicast discovery.</p>
+</li>
+<li>
+<p>Brittle to timing, especially on virtualized clusters where clocks are unpredictable.</p>
+</li>
+</ul>
+<p>It proved good for workload sharing -tasks can be published to it, any
+agent can say "I'm working on it" and take up the work. If the process
+fails, the task becomes available again. We used this for distributed scheduling in a rendering farm.</p>
+<h2 id="carmen"><a href="http://www.hpl.hp.com/techreports/2002/HPL-2002-257">Carmen</a></h2>
+<p>This was another HP Labs project, related to the Cooltown "ubiquitous
+computing" work, which was a decade too early to be relevant. It was
+also positioned by management as a B2B platform, so ended up competing
+with - and losing against - WS-* and UDDI. </p>
+<p>Carmen aimed to provide service discovery with both fixed services, and
+with highly mobile client services that will roam around the network -they
+are assumed to be wireless devices.</p>
+<p>Services were published with and searched for by attributed, locality
+was considered to be a key attribute -local instances of a service
+prioritized. Those services with a static location and low rate of
+change became the stable caches of service information -becoming,
+as with skype, "supernodes". </p>
+<p>Bootstrapping the cluster relied on multicast, though alternatives
+based on DHCP and DNS were proposed.</p>
+ </div>
+
+ <div id="footer">
+ <a alt="Apache Software Foundation" href="http://www.apache.org">
+ <img id="asf-logo" alt="Apache Software Foundation" src="/images/feather-small.gif" width="100">
+ </a>
+ <div class="copyright">
+ <p>
+ Copyright © 2011-2014 The Apache Software Foundation, Licensed under
+ the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+ Apache Slider, Slider, Apache, the Apache feather logo, and the Apache Incubator
+ project logo are trademarks of the <a href="http://www.apache.org">Apache Software Foundation</a>.<br />
+ Site created with <a href="http://getbootstrap.com/">Bootstrap</a> including icons from <a href="http://glyphicons.com/">GLYPHICONS</a> and <a href="http://fontawesome.io/">Font Awesome</a>.
+ </p>
+ </div>
+ </div>
+ </div>
+
+</div>
+</div>
+
+<script type="text/javascript">
+
+</script>
+</body>
+</html>
Added: websites/staging/slider/trunk/content/design/registry/references.html
==============================================================================
--- websites/staging/slider/trunk/content/design/registry/references.html (added)
+++ websites/staging/slider/trunk/content/design/registry/references.html Tue Jul 22 18:17:12 2014
@@ -0,0 +1,236 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href="/css/bootstrap.min.css" rel="stylesheet">
+ <link href="/css/bootstrap-theme.min.css" rel="stylesheet">
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
+ <script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <link href="/css/slider.css" rel="stylesheet" type="text/css">
+ <link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
+ <title></title>
+
+</head>
+
+<body>
+<nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+<div class="container-fluid">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-items">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/index.html">Slider</a>
+ </div>
+ <div class="collapse navbar-collapse" id="navbar-items">
+ <ul class="nav navbar-nav">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Project <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_index"><a href="/index.html">Home</a></li>
+<li id="nav_downloads"><a href="/downloads">Downloads</a></li>
+<li id="nav_source"><a href="https://git-wip-us.apache.org/repos/asf/incubator-slider.git"><i class="fa fa-external-link"></i> Source</a></li>
+<li><a href="http://www.apache.org/licenses/LICENSE-2.0"><i class="fa fa-external-link"></i> License</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Community</li>
+<li id="nav_team"><a href="/team.html">Team</a></li>
+<li id="nav_mailinglists"><a href="/mailing_lists.html">Mailing Lists</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Tools</li>
+<li id="nav_jira"><a href="https://issues.apache.org/jira/browse/SLIDER"><i class="fa fa-external-link"></i> Issues</a></li>
+<li id="nav_builds"><a href="https://builds.apache.org/job/Slider-develop"><i class="fa fa-external-link"></i> Builds</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/design">
+ Design <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_design_overview"><a href="/design/architecture.html">Overview</a></li>
+<li id="nav_app_needs"><a href="/docs/slider_specs/application_needs.html">Application Needs</a></li>
+<li id="nav_spec"><a href="/design/specification/index.html">Expected Behavior</a></li>
+<li id="nav_registry"><a href="/design/registry/index.html">Service Registry</a></li>
+<li id="nav_ssl_impl"><a href="/design/ssl_implementation.html">SSL Implementation</a></li>
+<li id="nav_role_history"><a href="/design/rolehistory.html">Role History</a></li>
+</ul>
+</li>
+
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/developing">
+ Developing <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_dev_building"><a href="/developing/building.html">Building</a></li>
+<li id="nav_dev_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_dev_testing"><a href="/developing/testing.html">Testing</a></li>
+<li id="nav_dev_funtesting"><a href="/developing/functional_tests.html">Functional Testing</a></li>
+<li id="nav_dev_manual_testing"><a href="/developing/manual_testing.html">Manual Testing</a></li>
+<li id="nav_releasing"><a href="/developing/releasing.html">Releasing</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Using <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_use_getting_started"><a href="/docs/getting_started.html">Getting Started</a></li>
+<li id="nav_use_manpage"><a href="/docs/manpage.html">Man Page</a></li>
+<li id="nav_use_examples"><a href="/docs/examples.html">Examples</a></li>
+<li id="nav_use_client_config"><a href="/docs/client-configuration.html">Client Configuration</a></li>
+<li id="nav_use_exitcodes"><a href="/docs/exitcodes.html">Client Exit Codes</a></li>
+<li id="nav_use_security"><a href="/docs/security.html">Security</a></li>
+<li id="nav_am_agent_ssl"><a href="/docs/ssl.html">Agent to AM SSL</a></li>
+<li id="nav_use_app_package"><a href="/docs/slider_specs/index.html">Creating App Packages</a></li>
+<li id="nav_use_app_configuration"><a href="/docs/configuration/index.html">App Configuration Model</a></li>
+<li id="nav_use_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_use_troubleshooting"><a href="/docs/troubleshooting.html">Troubleshooting</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ ASF Links <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li><a href="http://www.apache.org"><i class="fa fa-external-link"></i> Apache Software Foundation</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html"><i class="fa fa-external-link"></i> Sponsorship</a></li>
+<li><a href="http://www.apache.org/security/"><i class="fa fa-external-link"></i> Security</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html"><i class="fa fa-external-link"></i> Thanks</a></li>
+</ul>
+</li>
+</ul>
+
+ </div>
+
+</div>
+</nav>
+
+
+<div class="container-fluid">
+<div class="row">
+ <div class="col-md-2" id="sidebar">
+ <div style="text-align: center">
+ <h1><a href="/index.html">Apache Slider (incubating)</a></h1>
+ <hr>
+Latest release: <strong>0.40-incubating</strong><br>
+ <br>
+ <a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
+ </div>
+ <hr>
+ <a href="http://incubator.apache.org"><img id="logo" alt="Apache Incubator ™" class="img-responsive" src="http://incubator.apache.org/images/apache-incubator-logo.png"/></a>
+ </div>
+ <div class="col-md-8 col-md-offset-1">
+
+ <div id="content">
+
+ <h1 class="title"></h1>
+
+ <!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<h1 id="service-registry-references">Service Registry References</h1>
+<p>Service registration and discovery is a problem in distributed computing that has been explored for over thirty years, with
+[Birrell81]'s <em>Grapevine</em> system the first known implementation -though of </p>
+<h1 id="papers">Papers</h1>
+<ul>
+<li>
+<p><strong>[Birrell81]</strong> Birrell, A. et al, <a href="http://research.microsoft.com/apps/pubs/default.aspx?id=63661"><em>Grapevine: An exercise in distributed computing</em></a>. Comm. ACM 25, 4 (Apr 1982), pp260-274.
+The first documented directory service; relied on service shutdown to resolve update operations.</p>
+</li>
+<li>
+<p><strong>[Das02]</strong> <a href="http://www.cs.cornell.edu/~asdas/research/dsn02-swim.pdf"><em>SWIM: Scalable Weakly-consistent Infection-style Process Group Membership Protocol</em></a>
+P2P gossip-style data sharing protocol with random liveness probes to address scalable liveness checking. Ceph uses similar liveness checking.</p>
+</li>
+<li>
+<p><strong>[Marti02]</strong> Marti S. and Krishnam V., <a href="http://www.hpl.hp.com/techreports/2002/HPL-2002-257"><em>Carmen: A Dynamic Service Discovery Architecture</em></a>, </p>
+</li>
+<li>
+<p><strong>[Lampson86]</strong> Lampson, B. <a href="http://research.microsoft.com/en-us/um/people/blampson/36-GlobalNames/Acrobat.pdf"><em>Designing a Global Naming Service</em></a>. DEC.
+Distributed; includes an update protocol and the ability to add links to other parts of the tree. Also refers to <a href="http://bitsavers.informatik.uni-stuttgart.de/pdf/xerox/parc/techReports/OPD-T8103_The_Clearinghouse.pdf"><em>Xerox Clearinghouse</em></a>, which apparently shipped.</p>
+</li>
+<li>
+<p><strong>[Mockapetris88]</strong> Mockapetris, P. <a href="http://bnrg.eecs.berkeley.edu/~randy/Courses/CS268.F08/papers/31_dns.pdf"><em>Development of the domain name system</em></a>. The history of DNS</p>
+</li>
+<li>
+<p><strong>[Schroeder84]</strong> Schroeder, M.D. et al, <a href="http://research.microsoft.com/apps/pubs/default.aspx?id=61509"><em>Experience with Grapevine: The Growth of a Distributed System</em></a>. Xerox.
+Writeup of the experiences of using grapevine, with its eventual consistency and lack of idempotent message delivery called out -along with coverage of operations issues.</p>
+</li>
+<li>
+<p><strong>[van Renesse08]</strong> van Renesse, R. et al, <a href="http://www.cs.cornell.edu/home/rvr/papers/astrolabe.pdf"><em>Astrolabe: A Robust and Scalable Technology For Distributed System Monitoring, Management, and Data Mining</em></a>. ACM Transactions on Computer Systems
+Grandest P2P management framework to date; the work that earned Werner Vogel his CTO position at Amazon.</p>
+</li>
+<li>
+<p><strong>[van Steen86]</strong> van Steen, M. et al, <a href="http://www.cs.vu.nl/~ast/publications/asci-1996a.pdf"><em>A Scalable Location Service for Distributed Objects</em></a>.
+Vrije Universiteit, Amsterdam. Probably the first Object Request Broker</p>
+</li>
+</ul>
+ </div>
+
+ <div id="footer">
+ <a alt="Apache Software Foundation" href="http://www.apache.org">
+ <img id="asf-logo" alt="Apache Software Foundation" src="/images/feather-small.gif" width="100">
+ </a>
+ <div class="copyright">
+ <p>
+ Copyright © 2011-2014 The Apache Software Foundation, Licensed under
+ the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+ Apache Slider, Slider, Apache, the Apache feather logo, and the Apache Incubator
+ project logo are trademarks of the <a href="http://www.apache.org">Apache Software Foundation</a>.<br />
+ Site created with <a href="http://getbootstrap.com/">Bootstrap</a> including icons from <a href="http://glyphicons.com/">GLYPHICONS</a> and <a href="http://fontawesome.io/">Font Awesome</a>.
+ </p>
+ </div>
+ </div>
+ </div>
+
+</div>
+</div>
+
+<script type="text/javascript">
+
+</script>
+</body>
+</html>
Added: websites/staging/slider/trunk/content/design/registry/registry-model.html
==============================================================================
--- websites/staging/slider/trunk/content/design/registry/registry-model.html (added)
+++ websites/staging/slider/trunk/content/design/registry/registry-model.html Tue Jul 22 18:17:12 2014
@@ -0,0 +1,272 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href="/css/bootstrap.min.css" rel="stylesheet">
+ <link href="/css/bootstrap-theme.min.css" rel="stylesheet">
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
+ <script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <link href="/css/slider.css" rel="stylesheet" type="text/css">
+ <link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
+ <title></title>
+
+</head>
+
+<body>
+<nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+<div class="container-fluid">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-items">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/index.html">Slider</a>
+ </div>
+ <div class="collapse navbar-collapse" id="navbar-items">
+ <ul class="nav navbar-nav">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Project <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_index"><a href="/index.html">Home</a></li>
+<li id="nav_downloads"><a href="/downloads">Downloads</a></li>
+<li id="nav_source"><a href="https://git-wip-us.apache.org/repos/asf/incubator-slider.git"><i class="fa fa-external-link"></i> Source</a></li>
+<li><a href="http://www.apache.org/licenses/LICENSE-2.0"><i class="fa fa-external-link"></i> License</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Community</li>
+<li id="nav_team"><a href="/team.html">Team</a></li>
+<li id="nav_mailinglists"><a href="/mailing_lists.html">Mailing Lists</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Tools</li>
+<li id="nav_jira"><a href="https://issues.apache.org/jira/browse/SLIDER"><i class="fa fa-external-link"></i> Issues</a></li>
+<li id="nav_builds"><a href="https://builds.apache.org/job/Slider-develop"><i class="fa fa-external-link"></i> Builds</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/design">
+ Design <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_design_overview"><a href="/design/architecture.html">Overview</a></li>
+<li id="nav_app_needs"><a href="/docs/slider_specs/application_needs.html">Application Needs</a></li>
+<li id="nav_spec"><a href="/design/specification/index.html">Expected Behavior</a></li>
+<li id="nav_registry"><a href="/design/registry/index.html">Service Registry</a></li>
+<li id="nav_ssl_impl"><a href="/design/ssl_implementation.html">SSL Implementation</a></li>
+<li id="nav_role_history"><a href="/design/rolehistory.html">Role History</a></li>
+</ul>
+</li>
+
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/developing">
+ Developing <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_dev_building"><a href="/developing/building.html">Building</a></li>
+<li id="nav_dev_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_dev_testing"><a href="/developing/testing.html">Testing</a></li>
+<li id="nav_dev_funtesting"><a href="/developing/functional_tests.html">Functional Testing</a></li>
+<li id="nav_dev_manual_testing"><a href="/developing/manual_testing.html">Manual Testing</a></li>
+<li id="nav_releasing"><a href="/developing/releasing.html">Releasing</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Using <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_use_getting_started"><a href="/docs/getting_started.html">Getting Started</a></li>
+<li id="nav_use_manpage"><a href="/docs/manpage.html">Man Page</a></li>
+<li id="nav_use_examples"><a href="/docs/examples.html">Examples</a></li>
+<li id="nav_use_client_config"><a href="/docs/client-configuration.html">Client Configuration</a></li>
+<li id="nav_use_exitcodes"><a href="/docs/exitcodes.html">Client Exit Codes</a></li>
+<li id="nav_use_security"><a href="/docs/security.html">Security</a></li>
+<li id="nav_am_agent_ssl"><a href="/docs/ssl.html">Agent to AM SSL</a></li>
+<li id="nav_use_app_package"><a href="/docs/slider_specs/index.html">Creating App Packages</a></li>
+<li id="nav_use_app_configuration"><a href="/docs/configuration/index.html">App Configuration Model</a></li>
+<li id="nav_use_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_use_troubleshooting"><a href="/docs/troubleshooting.html">Troubleshooting</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ ASF Links <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li><a href="http://www.apache.org"><i class="fa fa-external-link"></i> Apache Software Foundation</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html"><i class="fa fa-external-link"></i> Sponsorship</a></li>
+<li><a href="http://www.apache.org/security/"><i class="fa fa-external-link"></i> Security</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html"><i class="fa fa-external-link"></i> Thanks</a></li>
+</ul>
+</li>
+</ul>
+
+ </div>
+
+</div>
+</nav>
+
+
+<div class="container-fluid">
+<div class="row">
+ <div class="col-md-2" id="sidebar">
+ <div style="text-align: center">
+ <h1><a href="/index.html">Apache Slider (incubating)</a></h1>
+ <hr>
+Latest release: <strong>0.40-incubating</strong><br>
+ <br>
+ <a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
+ </div>
+ <hr>
+ <a href="http://incubator.apache.org"><img id="logo" alt="Apache Incubator ™" class="img-responsive" src="http://incubator.apache.org/images/apache-incubator-logo.png"/></a>
+ </div>
+ <div class="col-md-8 col-md-offset-1">
+
+ <div id="content">
+
+ <h1 class="title"></h1>
+
+ <!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<h1 id="apache-slider-service-registry">Apache Slider: Service Registry</h1>
+<p>The service registry model is designed to support dynamically
+deployed Slider applications, <em>and</em> statically deployed versions
+of the same application -provided the latter also registers itself,
+its public network services, and any configurations and files
+that it wishes clients to be able to retrieve.</p>
+<p>The architecture and implementation of this registry is not defined
+here -instead the view of it seen by clients.</p>
+<ol>
+<li>
+<p>A 'service registry' exists in the YARN cluster into which
+services can be registered. </p>
+</li>
+<li>
+<p>There is no restriction on the number of services that can be registered in
+the registry, the type of service that may register, or even on how many
+registered services an application running in the YARN cluster may register.</p>
+</li>
+<li>
+<p>Services are registered by their type, owner and name. As an example,
+Alice's slider-managed HBase cluster <code>ingress</code> would have a type <code>org.apache.hbase</code>,
+owner <code>alice</code> and name <code>ingress</code>. </p>
+</li>
+<li>
+<p>In the case of Slider-managed services, there is a separate slider instance
+registration which publishes information about slider itself. In the example
+above, this would be (<code>org.apache.slider</code>,<code>alice</code>,<code>ingress</code>).</p>
+</li>
+<li>
+<p>Services can publish information about themselves, with common entries being:</p>
+<ul>
+<li>service name and description.</li>
+<li>URLs of published web UIs and web service APIs</li>
+<li>network address of other protocols</li>
+</ul>
+</li>
+<li>
+<p>Services may also publish. </p>
+<ul>
+<li>URLs to configuration details</li>
+<li>URLs documents published for use as client-side configuration data -either
+ directly or through some form of processing.</li>
+<li>public service-specific data, for use by applications that are aware of
+ the specific service type.</li>
+<li>internal service-specific data -for use by the components that comprise
+ an application. This allows the registry to be used to glue together
+ the application itself.</li>
+</ul>
+</li>
+<li>
+<p>Services can be listed and examined.</p>
+</li>
+<li>
+<p>Service-published configuration key-value pairs can be retrieved by clients</p>
+</li>
+<li>
+<p>Service-published documents (and packages of such documents) can be
+retrieved by clients.</p>
+</li>
+<li>
+<p>There's no requirement for service instances to support any standard protocols;</p>
+</li>
+<li>
+<p>Some protocols are defined which they MAY implement. For example, the protocol
+to enumerate and retrieve configuration documents is designed to be implemented
+by any service that wishes to publish such content.</p>
+</li>
+<li>
+<p>In a secure cluster, the registry data may be restricted, along with any
+service protocols offered by the registered services. </p>
+</li>
+</ol>
+ </div>
+
+ <div id="footer">
+ <a alt="Apache Software Foundation" href="http://www.apache.org">
+ <img id="asf-logo" alt="Apache Software Foundation" src="/images/feather-small.gif" width="100">
+ </a>
+ <div class="copyright">
+ <p>
+ Copyright © 2011-2014 The Apache Software Foundation, Licensed under
+ the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+ Apache Slider, Slider, Apache, the Apache feather logo, and the Apache Incubator
+ project logo are trademarks of the <a href="http://www.apache.org">Apache Software Foundation</a>.<br />
+ Site created with <a href="http://getbootstrap.com/">Bootstrap</a> including icons from <a href="http://glyphicons.com/">GLYPHICONS</a> and <a href="http://fontawesome.io/">Font Awesome</a>.
+ </p>
+ </div>
+ </div>
+ </div>
+
+</div>
+</div>
+
+<script type="text/javascript">
+
+</script>
+</body>
+</html>
Added: websites/staging/slider/trunk/content/design/registry/service_registry_end_to_end_scenario.html
==============================================================================
--- websites/staging/slider/trunk/content/design/registry/service_registry_end_to_end_scenario.html (added)
+++ websites/staging/slider/trunk/content/design/registry/service_registry_end_to_end_scenario.html Tue Jul 22 18:17:12 2014
@@ -0,0 +1,393 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href="/css/bootstrap.min.css" rel="stylesheet">
+ <link href="/css/bootstrap-theme.min.css" rel="stylesheet">
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
+ <script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <link href="/css/slider.css" rel="stylesheet" type="text/css">
+ <link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
+ <title></title>
+
+</head>
+
+<body>
+<nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+<div class="container-fluid">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-items">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/index.html">Slider</a>
+ </div>
+ <div class="collapse navbar-collapse" id="navbar-items">
+ <ul class="nav navbar-nav">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Project <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_index"><a href="/index.html">Home</a></li>
+<li id="nav_downloads"><a href="/downloads">Downloads</a></li>
+<li id="nav_source"><a href="https://git-wip-us.apache.org/repos/asf/incubator-slider.git"><i class="fa fa-external-link"></i> Source</a></li>
+<li><a href="http://www.apache.org/licenses/LICENSE-2.0"><i class="fa fa-external-link"></i> License</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Community</li>
+<li id="nav_team"><a href="/team.html">Team</a></li>
+<li id="nav_mailinglists"><a href="/mailing_lists.html">Mailing Lists</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Tools</li>
+<li id="nav_jira"><a href="https://issues.apache.org/jira/browse/SLIDER"><i class="fa fa-external-link"></i> Issues</a></li>
+<li id="nav_builds"><a href="https://builds.apache.org/job/Slider-develop"><i class="fa fa-external-link"></i> Builds</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/design">
+ Design <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_design_overview"><a href="/design/architecture.html">Overview</a></li>
+<li id="nav_app_needs"><a href="/docs/slider_specs/application_needs.html">Application Needs</a></li>
+<li id="nav_spec"><a href="/design/specification/index.html">Expected Behavior</a></li>
+<li id="nav_registry"><a href="/design/registry/index.html">Service Registry</a></li>
+<li id="nav_ssl_impl"><a href="/design/ssl_implementation.html">SSL Implementation</a></li>
+<li id="nav_role_history"><a href="/design/rolehistory.html">Role History</a></li>
+</ul>
+</li>
+
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/developing">
+ Developing <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_dev_building"><a href="/developing/building.html">Building</a></li>
+<li id="nav_dev_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_dev_testing"><a href="/developing/testing.html">Testing</a></li>
+<li id="nav_dev_funtesting"><a href="/developing/functional_tests.html">Functional Testing</a></li>
+<li id="nav_dev_manual_testing"><a href="/developing/manual_testing.html">Manual Testing</a></li>
+<li id="nav_releasing"><a href="/developing/releasing.html">Releasing</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Using <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_use_getting_started"><a href="/docs/getting_started.html">Getting Started</a></li>
+<li id="nav_use_manpage"><a href="/docs/manpage.html">Man Page</a></li>
+<li id="nav_use_examples"><a href="/docs/examples.html">Examples</a></li>
+<li id="nav_use_client_config"><a href="/docs/client-configuration.html">Client Configuration</a></li>
+<li id="nav_use_exitcodes"><a href="/docs/exitcodes.html">Client Exit Codes</a></li>
+<li id="nav_use_security"><a href="/docs/security.html">Security</a></li>
+<li id="nav_am_agent_ssl"><a href="/docs/ssl.html">Agent to AM SSL</a></li>
+<li id="nav_use_app_package"><a href="/docs/slider_specs/index.html">Creating App Packages</a></li>
+<li id="nav_use_app_configuration"><a href="/docs/configuration/index.html">App Configuration Model</a></li>
+<li id="nav_use_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_use_troubleshooting"><a href="/docs/troubleshooting.html">Troubleshooting</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ ASF Links <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li><a href="http://www.apache.org"><i class="fa fa-external-link"></i> Apache Software Foundation</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html"><i class="fa fa-external-link"></i> Sponsorship</a></li>
+<li><a href="http://www.apache.org/security/"><i class="fa fa-external-link"></i> Security</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html"><i class="fa fa-external-link"></i> Thanks</a></li>
+</ul>
+</li>
+</ul>
+
+ </div>
+
+</div>
+</nav>
+
+
+<div class="container-fluid">
+<div class="row">
+ <div class="col-md-2" id="sidebar">
+ <div style="text-align: center">
+ <h1><a href="/index.html">Apache Slider (incubating)</a></h1>
+ <hr>
+Latest release: <strong>0.40-incubating</strong><br>
+ <br>
+ <a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
+ </div>
+ <hr>
+ <a href="http://incubator.apache.org"><img id="logo" alt="Apache Incubator ™" class="img-responsive" src="http://incubator.apache.org/images/apache-incubator-logo.png"/></a>
+ </div>
+ <div class="col-md-8 col-md-offset-1">
+
+ <div id="content">
+
+ <h1 class="title"></h1>
+
+ <!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<h1 id="apache-slider-service-registry-end-to-end-scenarios">Apache Slider: Service Registry End-to-End Scenarios</h1>
+<h2 id="am-startup">AM startup</h2>
+<ol>
+<li>
+<p>AM starts, reads in configuration, creates provider</p>
+</li>
+<li>
+<p>AM builds web site, involving provider in process (<em>there's a possible race condition here, due to the AM registration sequence)</em></p>
+</li>
+<li>
+<p>AM registers self with RM, including web and IPC ports, and receives list of existing containers; container loss notifications come in asynchronously <em>(which is why the AM startup process is in a synchronized block)</em></p>
+</li>
+<li>
+<p>AM inits it's <code>ApplicationState</code> instance with the config, instance description and RM-supplied container list.</p>
+</li>
+<li>
+<p>AM creates service registry client using ZK quorum and path provided when AM was started</p>
+</li>
+<li>
+<p>AM registers standard endpoints: RPC, WebUI, REST APIs</p>
+</li>
+<li>
+<p>AM registers standard content it can serve (e.g <code>yarn-site.xml</code>)</p>
+</li>
+<li>
+<p>AM passes registry to provider in <code>bind()</code> operation.</p>
+</li>
+<li>
+<p>AM triggers review of application state, requesting/releasing nodes as appropriate</p>
+</li>
+</ol>
+<h2 id="agent-startup-standalone">Agent Startup: standalone</h2>
+<ol>
+<li>
+<p>Container is issued to AM</p>
+</li>
+<li>
+<p>AM chooses component, launches agent on it -with URL of AM a parameter (TODO: Add registry bonding of ZK quorum and path)</p>
+</li>
+<li>
+<p>Agent starts up.</p>
+</li>
+<li>
+<p>Agent locates AM via URL/ZK info</p>
+</li>
+<li>
+<p>Agent heartbeats in with state</p>
+</li>
+<li>
+<p>AM gives agent next state command.</p>
+</li>
+</ol>
+<h2 id="am-gets-state-from-agent">AM gets state from agent:</h2>
+<ol>
+<li>
+<p>Agent heartbeats in</p>
+</li>
+<li>
+<p>AM decides if it wants to receive config </p>
+</li>
+<li>
+<p>AM issues request for state information -all (dynamic) config data</p>
+</li>
+<li>
+<p>Agent receives it</p>
+</li>
+<li>
+<p>Agent returns all config state, including: hostnames, allocated ports, generated values (e.g. database connection strings, URLs) - as two-level (allows agent to define which config options are relevant to which document)</p>
+</li>
+</ol>
+<h2 id="am-saves-state-for-serving">AM saves state for serving</h2>
+<ol>
+<li>
+<p>AM saves state in RAM (assumptions: small, will rebuild on restart)</p>
+</li>
+<li>
+<p>AM updates service registry with list of content that can be served up and URLs to retrieve them.</p>
+</li>
+<li>
+<p>AM fields HTTP GET requests on content</p>
+</li>
+</ol>
+<h2 id="am-serves-content">AM Serves content</h2>
+<p>A simple REST service serves up content on paths published to the service registry. It is also possible to enumerate documents published by GET operations on parent paths.</p>
+<ol>
+<li>
+<p>On GET command, AM locates referenced agent values</p>
+</li>
+<li>
+<p>AM builds up response document from K-V pairs. This can be in a limited set of formats: Hadoop XML, Java properties, YAML, CSV, HTTP, JSON chosen as ? type param. (this generation is done from template processing in AM using slider.core.template module)</p>
+</li>
+<li>
+<p>response is streamed with headers of : <code>content-type</code>, <code>content-length</code>, do not cache in proxy, expires,<em> (with expiry date chosen as ??)</em></p>
+</li>
+</ol>
+<h1 id="slider-client">Slider Client</h1>
+<p>Currently slider client enumerates the YARN registry looking for slider instances -including any instances of the same application running before launching a cluster. </p>
+<p>This </p>
+<ul>
+<li>has race conditions</li>
+<li>has scale limitations <code>O(apps-in-YARN-cluster)</code> + <code>O(completed-apps-in-RM-memory)</code></li>
+<li>only retrieves configuration information from slider-deployed application instances. <em>We do not need to restrict ourselves here.</em></li>
+</ul>
+<h2 id="slider-client-lists-applications">Slider Client lists applications</h2>
+<div class="codehilite"><pre><span class="n">slider</span> <span class="n">registry</span> <span class="o">--</span><span class="n">list</span> <span class="p">[</span><span class="o">--</span><span class="n">servicetype</span> <span class="o"><</span><span class="n">application</span><span class="o">-</span><span class="n">type</span><span class="o">></span><span class="p">]</span>
+</pre></div>
+
+
+<ol>
+<li>
+<p>Client starts</p>
+</li>
+<li>
+<p>Client creates creates service registry client using ZK quorum and path provided in client config properties (slider-client.xml)</p>
+</li>
+<li>
+<p>Client enumerates registered services and lists them</p>
+</li>
+</ol>
+<h2 id="slider-client-lists-content-published-by-an-application-instance">Slider Client lists content published by an application instance</h2>
+<div class="codehilite"><pre><span class="n">slider</span> <span class="n">registry</span> <span class="o"><</span><span class="n">instance</span><span class="o">></span> <span class="o">--</span><span class="n">listconf</span> <span class="p">[</span><span class="o">--</span><span class="n">servicetype</span> <span class="o"><</span><span class="n">application</span><span class="o">-</span><span class="n">type</span><span class="o">></span><span class="p">]</span>
+</pre></div>
+
+
+<ol>
+<li>
+<p>Client starts</p>
+</li>
+<li>
+<p>Client creates creates service registry client using ZK quorum and path provided in client config properties (slider-client.xml)</p>
+</li>
+<li>
+<p>Client locates registered service entry -or fails</p>
+</li>
+<li>
+<p>Client retrieves service data, specifically the listing of published documents</p>
+</li>
+<li>
+<p>Client displays list of content</p>
+</li>
+</ol>
+<h2 id="slider-client-retrieves-content-published-by-an-application-instance">Slider Client retrieves content published by an application instance</h2>
+<div class="codehilite"><pre><span class="n">slider</span> <span class="n">registry</span> <span class="o"><</span><span class="n">instance</span><span class="o">></span> <span class="o">--</span><span class="n">getconf</span> <span class="o"><</span><span class="n">document</span><span class="o">></span> <span class="p">[</span><span class="o">--</span><span class="n">format</span> <span class="p">(</span><span class="n">xml</span><span class="o">|</span><span class="k">properties</span><span class="o">|</span><span class="n">text</span><span class="o">|</span><span class="n">html</span><span class="o">|</span><span class="n">csv</span><span class="o">|</span><span class="n">yaml</span><span class="o">|</span><span class="n">json</span><span class="p">,...)</span> <span class="p">[</span><span class="o">--</span><span class="n">dest</span> <span class="o"><</span><span class="n">file</span><span class="o">></span><span class="p">]</span> <span class="p">[</span><
span class="o">--</span><span class="n">servicetype</span> <span class="o"><</span><span class="n">application</span><span class="o">-</span><span class="n">type</span><span class="o">></span><span class="p">]</span>
+</pre></div>
+
+
+<ol>
+<li>
+<p>Client starts</p>
+</li>
+<li>
+<p>Client creates creates service registry client using ZK quorum and path provided in client config properties (slider-client.xml)</p>
+</li>
+<li>
+<p>Client locates registered service entry -or fails</p>
+</li>
+<li>
+<p>Client retrieves service data, specifically the listing of published documents</p>
+</li>
+<li>
+<p>Client locates URL of content</p>
+</li>
+<li>
+<p>Client builds GET request including format</p>
+</li>
+<li>
+<p>Client executes command, follows redirects, validates content length against supplied data.</p>
+</li>
+<li>
+<p>Client prints response to console or saves to output file. This is the path specified as a destination, or, if that path refers to a directory, to
+a file underneath.</p>
+</li>
+</ol>
+<h2 id="slider-client-retrieves-content-set-published-by-an-application-instance">Slider Client retrieves content set published by an application instance</h2>
+<p>Here a set of documents published is retrieved in the desired format of an application.</p>
+<h2 id="slider-client-retrieves-document-and-applies-template-to-it">Slider Client retrieves document and applies template to it</h2>
+<p>Here a set of documents published is retrieved in the desired format of an application.</p>
+<div class="codehilite"><pre><span class="n">slider</span> <span class="n">registry</span> <span class="o"><</span><span class="n">instance</span><span class="o">></span> <span class="o">--</span><span class="n">source</span> <span class="o"><</span><span class="n">document</span><span class="o">></span> <span class="p">[</span><span class="o">--</span><span class="n">template</span> <span class="o"><</span><span class="n">path</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">template</span><span class="o">></span><span class="p">]</span> <span class="p">[</span><span class="o">--</span><span class="n">outfile</span> <span class="o"><</span><span class="n">file</span><span class="o">></span><span class="p">]</span> <span class="p">[</span><span class="o">--</span><span class="n">servicetype</span> <span class="o"><</span><span class="n">application</span><span class="o">-</span><span class="n">type</span><span
class="o">></span><span class="p">]</span>
+</pre></div>
+
+
+<ol>
+<li>
+<p>document is retrieved as before, using a simple format such as json to retrieve it.</p>
+</li>
+<li>
+<p>The document is parsed and converted back into K-V pairs</p>
+</li>
+<li>
+<p>A template using a common/defined template library is applied to the content , generating the final output.</p>
+</li>
+</ol>
+<p>Template paths may include local filesystem paths or (somehow) something in a package file</p>
+ </div>
+
+ <div id="footer">
+ <a alt="Apache Software Foundation" href="http://www.apache.org">
+ <img id="asf-logo" alt="Apache Software Foundation" src="/images/feather-small.gif" width="100">
+ </a>
+ <div class="copyright">
+ <p>
+ Copyright © 2011-2014 The Apache Software Foundation, Licensed under
+ the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+ Apache Slider, Slider, Apache, the Apache feather logo, and the Apache Incubator
+ project logo are trademarks of the <a href="http://www.apache.org">Apache Software Foundation</a>.<br />
+ Site created with <a href="http://getbootstrap.com/">Bootstrap</a> including icons from <a href="http://glyphicons.com/">GLYPHICONS</a> and <a href="http://fontawesome.io/">Font Awesome</a>.
+ </p>
+ </div>
+ </div>
+ </div>
+
+</div>
+</div>
+
+<script type="text/javascript">
+
+</script>
+</body>
+</html>
Added: websites/staging/slider/trunk/content/design/registry/the_YARN_application_registration_and_binding_problem.html
==============================================================================
--- websites/staging/slider/trunk/content/design/registry/the_YARN_application_registration_and_binding_problem.html (added)
+++ websites/staging/slider/trunk/content/design/registry/the_YARN_application_registration_and_binding_problem.html Tue Jul 22 18:17:12 2014
@@ -0,0 +1,386 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href="/css/bootstrap.min.css" rel="stylesheet">
+ <link href="/css/bootstrap-theme.min.css" rel="stylesheet">
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
+ <script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <link href="/css/slider.css" rel="stylesheet" type="text/css">
+ <link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
+ <title></title>
+
+</head>
+
+<body>
+<nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+<div class="container-fluid">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-items">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/index.html">Slider</a>
+ </div>
+ <div class="collapse navbar-collapse" id="navbar-items">
+ <ul class="nav navbar-nav">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Project <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_index"><a href="/index.html">Home</a></li>
+<li id="nav_downloads"><a href="/downloads">Downloads</a></li>
+<li id="nav_source"><a href="https://git-wip-us.apache.org/repos/asf/incubator-slider.git"><i class="fa fa-external-link"></i> Source</a></li>
+<li><a href="http://www.apache.org/licenses/LICENSE-2.0"><i class="fa fa-external-link"></i> License</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Community</li>
+<li id="nav_team"><a href="/team.html">Team</a></li>
+<li id="nav_mailinglists"><a href="/mailing_lists.html">Mailing Lists</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Tools</li>
+<li id="nav_jira"><a href="https://issues.apache.org/jira/browse/SLIDER"><i class="fa fa-external-link"></i> Issues</a></li>
+<li id="nav_builds"><a href="https://builds.apache.org/job/Slider-develop"><i class="fa fa-external-link"></i> Builds</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/design">
+ Design <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_design_overview"><a href="/design/architecture.html">Overview</a></li>
+<li id="nav_app_needs"><a href="/docs/slider_specs/application_needs.html">Application Needs</a></li>
+<li id="nav_spec"><a href="/design/specification/index.html">Expected Behavior</a></li>
+<li id="nav_registry"><a href="/design/registry/index.html">Service Registry</a></li>
+<li id="nav_ssl_impl"><a href="/design/ssl_implementation.html">SSL Implementation</a></li>
+<li id="nav_role_history"><a href="/design/rolehistory.html">Role History</a></li>
+</ul>
+</li>
+
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/developing">
+ Developing <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_dev_building"><a href="/developing/building.html">Building</a></li>
+<li id="nav_dev_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_dev_testing"><a href="/developing/testing.html">Testing</a></li>
+<li id="nav_dev_funtesting"><a href="/developing/functional_tests.html">Functional Testing</a></li>
+<li id="nav_dev_manual_testing"><a href="/developing/manual_testing.html">Manual Testing</a></li>
+<li id="nav_releasing"><a href="/developing/releasing.html">Releasing</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Using <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_use_getting_started"><a href="/docs/getting_started.html">Getting Started</a></li>
+<li id="nav_use_manpage"><a href="/docs/manpage.html">Man Page</a></li>
+<li id="nav_use_examples"><a href="/docs/examples.html">Examples</a></li>
+<li id="nav_use_client_config"><a href="/docs/client-configuration.html">Client Configuration</a></li>
+<li id="nav_use_exitcodes"><a href="/docs/exitcodes.html">Client Exit Codes</a></li>
+<li id="nav_use_security"><a href="/docs/security.html">Security</a></li>
+<li id="nav_am_agent_ssl"><a href="/docs/ssl.html">Agent to AM SSL</a></li>
+<li id="nav_use_app_package"><a href="/docs/slider_specs/index.html">Creating App Packages</a></li>
+<li id="nav_use_app_configuration"><a href="/docs/configuration/index.html">App Configuration Model</a></li>
+<li id="nav_use_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_use_troubleshooting"><a href="/docs/troubleshooting.html">Troubleshooting</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ ASF Links <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li><a href="http://www.apache.org"><i class="fa fa-external-link"></i> Apache Software Foundation</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html"><i class="fa fa-external-link"></i> Sponsorship</a></li>
+<li><a href="http://www.apache.org/security/"><i class="fa fa-external-link"></i> Security</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html"><i class="fa fa-external-link"></i> Thanks</a></li>
+</ul>
+</li>
+</ul>
+
+ </div>
+
+</div>
+</nav>
+
+
+<div class="container-fluid">
+<div class="row">
+ <div class="col-md-2" id="sidebar">
+ <div style="text-align: center">
+ <h1><a href="/index.html">Apache Slider (incubating)</a></h1>
+ <hr>
+Latest release: <strong>0.40-incubating</strong><br>
+ <br>
+ <a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
+ </div>
+ <hr>
+ <a href="http://incubator.apache.org"><img id="logo" alt="Apache Incubator ™" class="img-responsive" src="http://incubator.apache.org/images/apache-incubator-logo.png"/></a>
+ </div>
+ <div class="col-md-8 col-md-offset-1">
+
+ <div id="content">
+
+ <h1 class="title"></h1>
+
+ <!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<h1 id="apache-slider-yarn-application-registration-and-binding-the-problem">Apache Slider YARN Application Registration and Binding: the Problem</h1>
+<h2 id="march-2014">March 2014</h2>
+<h1 id="how-to-bind-client-applications-to-services-dynamically-placed-applications">How to bind client applications to services dynamically placed applications?</h1>
+<p>There are some constraints here</p>
+<ol>
+<li>
+<p>The clients may be running outside the cluster -potentially over long-haul links.</p>
+</li>
+<li>
+<p>The location of an application deployed in a YARN cluster cannot be predicted.</p>
+</li>
+<li>
+<p>The ports used for application service endpoints cannot be hard-coded
+or predicted. (Alternatively: if they are hard-coded, then Socket-In-Use exceptions may occur)</p>
+</li>
+</ol>
+<p>1: As components fail and get re-instantiated, their location may change.
+The rate of this depends on cluster and application stability; the longer
+ lived the application, the more common it is.</p>
+<p>Existing Hadoop client apps have a configuration problem of their own:
+how are the settings in files such as <code>yarn-site.xml</code>picked up by today's
+applications? This is an issue which has historically been out of scope
+for Hadoop clusters -but if we are looking at registration and binding
+of YARN applications, there should be no reason why
+static applications cannot be discovered and bonded to using the same mechanisms. </p>
+<h1 id="other-constraints">Other constraints:</h1>
+<ol>
+<li>
+<p>Reduce the amount of change needed in existing applications to a minimum
+---ideally none, though some pre-launch setup may be acceptable.</p>
+</li>
+<li>
+<p>Prevent malicious applications from registering a service endpoints.</p>
+</li>
+<li>
+<p>Scale with #of applications and #of clients; not overload on a cluster partitioning.</p>
+</li>
+<li>
+<p>Offer a design that works with apps that are deployed in a YARN custer
+outside of Slider. Rationale: want a mechanism that works with pure-YARN apps</p>
+</li>
+</ol>
+<h2 id="possible-solutions">Possible Solutions:</h2>
+<h3 id="zk">ZK</h3>
+<p>Client applications use ZK to find services (addresses #1, #2 and #3).
+Requires location code in the client.</p>
+<p>HBase and Accumulo do this as part of a failover-ready design.</p>
+<h3 id="dns">DNS</h3>
+<p>Client apps use DNS to find services, with custom DNS server for a
+subdomain representing YARN services. Addresses #1; with a shortened TTL and
+no DNS address caching, #3. #2 addressed only if other DNS entries are used to
+ publish service entries. </p>
+<p>Should support existing applications, with a configuration that is stable
+over time. It does require the clients to not cache DNS addresses forever
+(this must be explicitly set on Java applications,
+irrespective of the published TTL). It generates a load on the DNS servers
+that is <code>O(clients/TTL)</code></p>
+<p>Google Chubby offers a DNS service to handle this. ZK does not -yet.</p>
+<h3 id="floating-ip-addresses">Floating IP Addresses</h3>
+<p>If the clients know/cache IP addresses of services, these addresses could be
+floated across service instances. Linux HA has floating IP address support,
+while Docker containers can make use of them, especially if an integrated DHCP
+server handles the assignment of IP addresses to specific containers. </p>
+<p>ARP caching is the inevitable problem here, but it is still less brittle than
+relying on applications to know not to cache IP addresses -and nor does it
+place so much on DNS servers as short-TTL DNS entries.</p>
+<h3 id="ldap">LDAP</h3>
+<p>Enterprise Directory services are used to publish/locate services. Requires
+lookup into the directory on binding (#1, #2), re-lookup on failure (#3).
+LDAP permissions can prevent untrusted applications registering.</p>
+<ul>
+<li>
+<p>Works well with Windows registries.</p>
+</li>
+<li>
+<p>Less common Java-side, though possible -and implemented in the core Java
+libraries. Spring-LDAP is focused on connection to an LDAP server
+-not LDAP-driven application config.</p>
+</li>
+</ul>
+<h3 id="registration-web-service">Registration Web Service</h3>
+<p>Custom web service registration services used. </p>
+<ul>
+<li>
+<p>The sole WS-* one, UDDI, does not have a REST equivalent
+--DNS is assumed to take on that role.</p>
+</li>
+<li>
+<p>Requires new client-side code anyway.</p>
+</li>
+</ul>
+<h3 id="zookeeper-url-schema">Zookeeper URL Schema</h3>
+<p>Offer our own <code>zk://</code> URL; java & .NET implementations (others?) to resolve, browser plugins. </p>
+<ul>
+<li>
+<p>Would address requirements #1 & #3</p>
+</li>
+<li>
+<p>Cost: non-standard; needs an extension for every application/platform, and
+will not work with tools such as CURL or web browsers</p>
+</li>
+</ul>
+<h3 id="am-side-config-generation">AM-side config generation</h3>
+<p>App-side config generation-YARN applications to generate client-side
+configuration files for launch-time information (#1, #2).
+The AM can dynamically create these, and as the storage load is all in
+the AM, does not consume as much resources in a central server as would
+publishing it all to that central server.</p>
+<ul>
+<li>
+<p>Requires application to know of client-side applications to support -
+and be able to generate to their configuration information (i.e. formatted files).</p>
+</li>
+<li>
+<p>Requires the AM to get all information from deployed application components
+needed to generate bindings. Unless the AM can resolve YARN App templates,
+need a way to get one of the components in the app to generate settings for
+the entire cluster, and push them back.</p>
+</li>
+<li>
+<p>Needs to be repeated for all YARN apps, however deployed.</p>
+</li>
+<li>
+<p>Needs something similar for statically deployed applications.</p>
+</li>
+</ul>
+<h3 id="client-side-config-generation">Client-side config generation</h3>
+<p>YARN app to publish attributes as key-val pairs, client-side code to read and
+generate configs from (#1, #2). Example configuration generators could
+create: Hadoop-client XML, Spring, tomcat, guice configs, something for .NET.</p>
+<ul>
+<li>
+<p>Not limited to Hoya application deployments only.</p>
+</li>
+<li>
+<p>K-V pairs need to be published "somewhere". A structured section in the
+ZK tree per app is the obvious location -though potentially expensive. An
+alternative is AM-published data.</p>
+</li>
+<li>
+<p>Needs client-side code capable of extracting information from YARN cluster
+to generate client-specific configuration.</p>
+</li>
+<li>
+<p>Assumes (key, value) pairs sufficient for client config generation. Again,
+some template expansion will aid here (this time: client-side interpretation).</p>
+</li>
+<li>
+<p>Client config generators need to find and bind to the target application themselves.</p>
+</li>
+</ul>
+<p>Multiple options:</p>
+<ul>
+<li>
+<p>Standard ZK structure for YARN applications (maybe: YARN itself to register
+paths in ZK and set up child permissions,so enforcing security).</p>
+</li>
+<li>
+<p>Agents to push to ZK dynamic information as K-V pairs</p>
+</li>
+<li>
+<p>Agent provider on AM to fetch K-V pairs and include in status requests</p>
+</li>
+<li>
+<p>CLI to fetch app config keys, echo out responses (needs client log4j settings
+to log all slf/log4j to stderr, so that app > results.txt would save results explicitly</p>
+</li>
+<li>
+<p>client side code per app to generate specific binding information</p>
+</li>
+</ul>
+<h3 id="load-balancer-app-yarn-app">Load-balancer app Yarn App</h3>
+<p>Spread requests round a set of registered handlers, e.g web servers. Support
+plugins for session binding/sharding. </p>
+<p>Some web servers can do this already; a custom YARN app could use grizzy
+embedded. Binding problem exists, but would support scaleable dispatch of values.</p>
+<ul>
+<li>
+<p>Could be offered as an AM extension (in provider, ...): scales well
+with #of apps in cluster, but adds initial location/failover problems.</p>
+</li>
+<li>
+<p>If offered as a core-YARN service, location is handled via a fixed
+URL. This could place high load on the service, even just 302 redirects.</p>
+</li>
+</ul>
+ </div>
+
+ <div id="footer">
+ <a alt="Apache Software Foundation" href="http://www.apache.org">
+ <img id="asf-logo" alt="Apache Software Foundation" src="/images/feather-small.gif" width="100">
+ </a>
+ <div class="copyright">
+ <p>
+ Copyright © 2011-2014 The Apache Software Foundation, Licensed under
+ the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+ Apache Slider, Slider, Apache, the Apache feather logo, and the Apache Incubator
+ project logo are trademarks of the <a href="http://www.apache.org">Apache Software Foundation</a>.<br />
+ Site created with <a href="http://getbootstrap.com/">Bootstrap</a> including icons from <a href="http://glyphicons.com/">GLYPHICONS</a> and <a href="http://fontawesome.io/">Font Awesome</a>.
+ </p>
+ </div>
+ </div>
+ </div>
+
+</div>
+</div>
+
+<script type="text/javascript">
+
+</script>
+</body>
+</html>