You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by ct...@apache.org on 2016/10/26 19:37:54 UTC

[26/31] accumulo git commit: Jekyll build from gh-pages:899f3b1

http://git-wip-us.apache.org/repos/asf/accumulo/blob/56754f27/blog/2014/06/25/scaling-accumulo-with-multivolume-support.html
----------------------------------------------------------------------
diff --git a/blog/2014/06/25/scaling-accumulo-with-multivolume-support.html b/blog/2014/06/25/scaling-accumulo-with-multivolume-support.html
new file mode 100644
index 0000000..c6fe94b
--- /dev/null
+++ b/blog/2014/06/25/scaling-accumulo-with-multivolume-support.html
@@ -0,0 +1,352 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+    Licensed to the Apache Software Foundation (ASF) under one or more
+    contributor license agreements.  See the NOTICE file distributed with
+    this work for additional information regarding copyright ownership.
+    The ASF licenses this file to You under the Apache License, Version 2.0
+    (the "License"); you may not use this file except in compliance with
+    the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+-->
+<meta charset="utf-8">
+<meta http-equiv="X-UA-Compatible" content="IE=edge">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<link href="/css/bootstrap.min.css" rel="stylesheet">
+<link href="/css/accumulo.css" rel="stylesheet" type="text/css">
+<link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
+<link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/v/bs/jqc-1.12.3/dt-1.10.12/datatables.min.css"/>
+
+<title>Scaling Accumulo with Multi-Volume Support</title>
+
+<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+<script src="/js/bootstrap.min.js"></script>
+<script type="text/javascript" src="https://cdn.datatables.net/v/bs-3.3.6/jqc-1.12.3/dt-1.10.12/datatables.min.js"></script>
+
+<script>
+  // show location of canonical site if not currently on the canonical site
+  $(function() {
+    var host = window.location.host;
+    if (typeof host !== 'undefined' && host !== 'accumulo.apache.org') {
+      $('#non-canonical').show();
+    }
+  });
+
+  
+
+  $(function() {
+    // decorate section headers with anchors
+    return $("h2, h3, h4, h5, h6").each(function(i, el) {
+      var $el, icon, id;
+      $el = $(el);
+      id = $el.attr('id');
+      icon = '<i class="fa fa-link"></i>';
+      if (id) {
+        return $el.append($("<a />").addClass("header-link").attr("href", "#" + id).html(icon));
+      }
+    });
+  });
+
+  // configure Google Analytics
+  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+  })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+  if (ga.hasOwnProperty('loaded') && ga.loaded === true) {
+    ga('create', 'UA-50934829-1', 'apache.org');
+    ga('send', 'pageview');
+  }
+</script>
+</head>
+<body style="padding-top: 100px">
+
+  <nav class="navbar navbar-default navbar-fixed-top">
+  <div class="container">
+    <div class="navbar-header">
+      <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-items">
+        <span class="sr-only">Toggle navigation</span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+      </button>
+      <a href="/"><img id="nav-logo" alt="Apache Accumulo" class="img-responsive" src="/images/accumulo-logo.png" width="200"/></a>
+    </div>
+    <div class="collapse navbar-collapse" id="navbar-items">
+      <ul class="nav navbar-nav">
+        <li class="nav-link"><a href="/downloads">Download</a></li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li class="dropdown-header">1.8.0 Release (Latest)</li>
+          <li id="nav_rn_180"><a href="/release_notes/1.8.0">Release Notes</a></li>
+          <li><a href="/1.8/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.8/apidocs">Javadoc</a></li>
+          <li id="nav_examples_18"><a href="/1.8/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">1.7.2 Release</li>
+          <li id="nav_rn_172"><a href="/release_notes/1.7.2">Release Notes</a></li>
+          <li><a href="/1.7/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.7/apidocs">Javadoc</a></li>
+          <li id="nav_examples_17"><a href="/1.7/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">1.6.6 Release</li>
+          <li id="nav_rn_166"><a href="/release_notes/1.6.6">Release Notes</a></li>
+          <li><a href="/1.6/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.6/apidocs">Javadoc</a></li>
+          <li id="nav_examples_16"><a href="/1.6/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li id="nav_features"><a href="/notable_features">Features</a></li>
+          <li id="nav_rn"><a href="/release_notes">Release Notes Archive</a></li>
+          <li id="old_documentation"><a href="/old_documentation">Documentation Archive</a></li>
+          <li id="nav_screenshots"><a href="/screenshots">Screenshots</a></li>
+          <li id="nav_papers"><a href="/papers">Papers &amp; Presentations</a></li>
+          <li id="nav_glossary"><a href="/glossary">Glossary</a></li>
+        </ul>
+        </li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li id="nav_getinvolved"><a href="/get_involved">Get Involved</a></li>
+          <li id="nav_mailinglists"><a href="/mailing_list">Mailing Lists</a></li>
+          <li id="nav_people"><a href="/people">People</a></li>
+          <li id="nav_news"><a href="/news/">News Archive</a></li>
+          <li id="nav_projects"><a href="/projects">Community Projects</a></li>
+          <li id="nav_thanks"><a href="/thanks">Thanks</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">Governance</li>
+          <li id="nav_bylaws"><a href="/bylaws">Bylaws</a></li>
+          <li id="nav_consensusbuilding"><a href="/governance/consensusBuilding">Consensus Building</a></li>
+          <li id="nav_lazyconsensus"><a href="/governance/lazyConsensus">Lazy Consensus</a></li>
+          <li id="nav_govreleasing"><a href="/governance/releasing">Releasing</a></li>
+          <li id="nav_voting"><a href="/governance/voting">Voting</a></li>
+        </ul>
+        </li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Development<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li><a href="https://issues.apache.org/jira/browse/ACCUMULO">Issue Tracker <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://github.com/apache/accumulo/pulls">Pull Requests <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://builds.apache.org/view/A/view/Accumulo">Jenkins Builds <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://travis-ci.org/apache/accumulo">TravisCI Builds <i class="fa fa-external-link"></i></a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">Guides</li>
+          <li id="nav_source"><a href="/source">Source &amp; Guide</a></li>
+          <li id="nav_git"><a href="/git">Git Workflow</a></li>
+          <li id="nav_versioning"><a href="/versioning">Versioning</a></li>
+          <li id="nav_contrib"><a href="/contrib">Contrib Projects</a></li>
+          <li id="nav_rb"><a href="/rb">Review Board</a></li>
+          <li id="nav_releasing"><a href="/releasing">Making Releases</a></li>
+          <li id="nav_verify_release"><a href="/verifying_releases">Verifying Releases</a></li>
+        </ul>
+        </li>
+      </ul>
+      <ul class="nav navbar-nav navbar-right">
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li><a href="https://www.apache.org">Apache Homepage <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/security">Security <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/thanks">Thanks <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct <i class="fa fa-external-link"></i></a></li>
+        </ul>
+        </li>
+      </ul>
+    </div>
+  </div>
+</nav>
+
+
+  <div class="container">
+    <div class="row">
+      <div class="col-md-12">
+
+        <div id="non-canonical" style="display: none; background-color: #F0E68C; padding-left: 1em;">
+          Visit the official site at: <a href="https://accumulo.apache.org">https://accumulo.apache.org</a>
+        </div>
+        <div id="content">
+          
+          <h1 class="title">Scaling Accumulo with Multi-Volume Support</h1>
+          
+          <h5>Dave Marion & Eric Newton - 25 Jun 2014 </h5>
+
+<p>Originally posted at <a href="https://blogs.apache.org/accumulo/">https://blogs.apache.org/accumulo/</a></p>
+
+<p>MapReduce is a commonly used approach to querying or analyzing large amounts of data. Typically MapReduce jobs are created using using some set of files in HDFS to produce a result. When new files come in, they get added to the set, and the job gets run again. A common Accumulo approach to this scenario is to load all of the data into a single instance of Accumulo.</p>
+
+<p>A single instance of Accumulo can scale quite largely(<a href="http://ieeexplore.ieee.org/zpl/login.jsp?arnumber=6597155">1</a>, <a href="http://www.pdl.cmu.edu/SDI/2013/slides/big_graph_nsa_rd_2013_56002v1.pdf">2</a>) to accommodate high levels of ingest and query. The manner in which ingest is performed typically depends on latency requirements. When the desired latency is small, inserts are performed directly into Accumulo. When the desired latency is allowed to be large, then a <a href="http://accumulo.apache.org/1.6/examples/bulkIngest.html">bulk style of ingest</a> can be used. There are other factors to consider as well, but they are outside the scope of this article.</p>
+
+<p>On large clusters using the bulk style of ingest input files are typically batched into MapReduce jobs to create a set of output RFiles for import into Accumulo. The number of files per job is typically determined by the required latency and the number of MapReduce tasks that the cluster can complete in the given time-frame. The resulting RFiles, when imported into Accumulo, are added to the list of files for their associated tablets. Depending on the configuration this will cause Accumulo to major compact these tablets. If the configuration is tweaked to allow more files per tablet, to reduce the major compactions, then more files need to be opened at query time when performing scans on the tablet. Note that no single node is burdened by the file management; but, the number of file operations in aggregate is very large. If each server has several hundred tablets, and there are a thousand tablet servers, and each tablet compacts some files every few imports, we easily have 50,000
  file operations (create, allocate a block, rename and delete) every ingest cycle.</p>
+
+<p>In addition to the NameNode operations caused by bulk ingest, other Accumulo processes (e.g. master, gc) require interaction with the NameNode. Single processes, like the garbage collector, can be starved of responses from the NameNode as the NameNode is limited on the number of concurrent operations. It is not unusual for an operator\u2019s request for \u201chadoop fs -ls /accumulo\u201d to take a minute before returning results during the peak file-management periods. In particular, the file garbage collector can fall behind, not finishing a cycle of unreferenced file removal before the next ingest cycle creates a new batch of files to be deleted.</p>
+
+<p>The Hadoop community addressed the NameNode bottleneck issue with <a href="https://issues.apache.org/jira/browse/HDFS-1052">HDFS federation</a> which allows a datanode to serve up blocks for multiple namenodes. Additionally, ViewFS allows clients to communicate with multiple namenodes through the use of a client-side mount table. This functionality was insufficient for Accumulo in the 1.6.0 release as ViewFS works at a directory level; as an example, /dirA is mapped to one NameNode and /dirB is mapped to another, and Accumulo uses a single HDFS directory for its storage.</p>
+
+<p>Multi-Volume support (MVS), included in 1.6.0, includes the changes that allow Accumulo to work across multiple HDFS clusters (called volumes in Accumulo) while continuing to use a single HDFS directory. A new property, instance.volumes, can be configured with multiple HDFS nameservices and Accumulo will use them all to balance out NameNode operations. The nameservices configured in instance.volumes may optionally use the High Availability NameNode feature as it is transparent to Accumulo. With MVS you have two options to horizontally scale your Accumulo instance. You can use an HDFS cluster with Federation and multiple NameNodes or you can use separate HDFS clusters.</p>
+
+<p>By default Accumulo will perform round-robin file allocation for each tablet, spreading the files across the different volumes. The file balancer is pluggable, allowing for custom implementations. For example, if you don\u2019t use Federation and use multiple HDFS clusters, you may want to allocate all files for a particular table to one volume.</p>
+
+<p>Comments in the <a href="https://issues.apache.org/jira/browse/ACCUMULO-118">JIRA</a> regarding backups could lead to follow-on work. With the inclusion of snapshots in HDFS, you could easily envision an application that quiesces the database or some set of tables, flushes their entries from memory, and snapshots their directories. These snapshots could then be copied to another HDFS instance either for an on-disk backup, or bulk-imported into another instance of Accumulo for testing or some other use.</p>
+
+<p>The example configuration below shows how to set up Accumulo with HA NameNodes and Federation, as it is likely the most complex. We had to reference several web sites, one of the HDFS mailing lists, and the source code to find all of the configuration parameters that were needed. The configuration below includes two sets of HA namenodes, each set servicing an HDFS nameservice in a single HDFS cluster. In the example below, nameserviceA is serviced by name nodes 1 and 2, and nameserviceB is serviced by name nodes 3 and 4.</p>
+
+<h3 id="core-sitexml">core-site.xml</h3>
+
+<div class="language-xml highlighter-rouge"><pre class="highlight"><code><span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>fs.defaultFS<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>viewfs:///<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>fs.viewfs.mounttable.default.link./nameserviceA<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>hdfs://nameserviceA<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>fs.viewfs.mounttable.default.link./nameserviceB<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>hdfs://nameserviceB<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>fs.viewfs.mounttable.default.link./nameserviceA/accumulo/instance_id<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>hdfs://nameserviceA/accumulo/instance_id<span class="nt">&lt;/value&gt;</span>
+  <span class="nt">&lt;description&gt;</span>Workaround for ACCUMULO-2719<span class="nt">&lt;/description&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.ha.fencing.methods<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>sshfence(hdfs:22)      
+         shell(/bin/true)<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>   
+  <span class="nt">&lt;name&gt;</span>dfs.ha.fencing.ssh.private-key-files<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;&lt;PRIVATE_KEY_LOCATION&gt;&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.ha.fencing.ssh.connect-timeout<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>30000<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>ha.zookeeper.quorum<span class="nt">&lt;/name&gt;</span>   
+  <span class="nt">&lt;value&gt;</span>zkHost1:2181,zkHost2:2181,zkHost3:2181<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+</code></pre>
+</div>
+
+<h3 id="hdfs-sitexml">hdfs-site.xml</h3>
+
+<div class="language-xml highlighter-rouge"><pre class="highlight"><code><span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.nameservices<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>nameserviceA,nameserviceB<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.ha.namenodes.nameserviceA<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>nn1,nn2<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span> 
+  <span class="nt">&lt;name&gt;</span>dfs.ha.namenodes.nameserviceB<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>nn3,nn4<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.rpc-address.nameserviceA.nn1<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>host1:8020<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.rpc-address.nameserviceA.nn2<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>host2:8020<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.http-address.nameserviceA.nn1<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>host1:50070<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.http-address.nameserviceA.nn2<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>host2:50070<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.rpc-address.nameserviceB.nn3<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>host3:8020<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.rpc-address.nameserviceB.nn4<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>host4:8020<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span> 
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.http-address.nameserviceB.nn3<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>host3:50070<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.http-address.nameserviceB.nn4<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>host4:50070<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span> 
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.shared.edits.dir.nameserviceA.nn1<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>qjournal://jHost1:8485;jHost2:8485;jHost3:8485/nameserviceA<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.shared.edits.dir.nameserviceA.nn2<span class="nt">&lt;/name&gt;</span>   
+  <span class="nt">&lt;value&gt;</span>qjournal://jHost1:8485;jHost2:8485;jHost3:8485/nameserviceA<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.shared.edits.dir.nameserviceB.nn3<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>qjournal://jHost1:8485;jHost2:8485;jHost3:8485/nameserviceB<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.namenode.shared.edits.dir.nameserviceB.nn4<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>qjournal://jHost1:8485;jHost2:8485;jHost3:8485/nameserviceB<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.client.failover.proxy.provider.nameserviceA<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span> 
+  <span class="nt">&lt;name&gt;</span>dfs.client.failover.proxy.provider.nameserviceB<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.ha.automatic-failover.enabled.nameserviceA<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>true<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>dfs.ha.automatic-failover.enabled.nameserviceB<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>true<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+</code></pre>
+</div>
+
+<h3 id="accumulo-sitexml">accumulo-site.xml</h3>
+
+<div class="language-xml highlighter-rouge"><pre class="highlight"><code><span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>instance.volumes<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>hdfs://nameserviceA/accumulo,hdfs://nameserviceB/accumulo<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+</code></pre>
+</div>
+
+
+
+<p><strong>View all posts in the <a href="/news/">news archive</a></strong></p>
+
+        </div>
+
+        
+<footer>
+
+  <p><a href="https://www.apache.org"><img src="/images/feather-small.gif" alt="Apache Software Foundation" id="asf-logo" height="100" /></a></p>
+
+  <p>Copyright � 2011-2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache�License,�Version�2.0</a>.</p>
+
+</footer>
+
+
+      </div>
+    </div>
+  </div>
+</body>
+</html>

http://git-wip-us.apache.org/repos/asf/accumulo/blob/56754f27/blog/2014/07/09/functional-reads-over-accumulo.html
----------------------------------------------------------------------
diff --git a/blog/2014/07/09/functional-reads-over-accumulo.html b/blog/2014/07/09/functional-reads-over-accumulo.html
new file mode 100644
index 0000000..6914fd3
--- /dev/null
+++ b/blog/2014/07/09/functional-reads-over-accumulo.html
@@ -0,0 +1,255 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+    Licensed to the Apache Software Foundation (ASF) under one or more
+    contributor license agreements.  See the NOTICE file distributed with
+    this work for additional information regarding copyright ownership.
+    The ASF licenses this file to You under the Apache License, Version 2.0
+    (the "License"); you may not use this file except in compliance with
+    the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+-->
+<meta charset="utf-8">
+<meta http-equiv="X-UA-Compatible" content="IE=edge">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<link href="/css/bootstrap.min.css" rel="stylesheet">
+<link href="/css/accumulo.css" rel="stylesheet" type="text/css">
+<link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
+<link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/v/bs/jqc-1.12.3/dt-1.10.12/datatables.min.css"/>
+
+<title>Functional reads over Accumulo</title>
+
+<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+<script src="/js/bootstrap.min.js"></script>
+<script type="text/javascript" src="https://cdn.datatables.net/v/bs-3.3.6/jqc-1.12.3/dt-1.10.12/datatables.min.js"></script>
+
+<script>
+  // show location of canonical site if not currently on the canonical site
+  $(function() {
+    var host = window.location.host;
+    if (typeof host !== 'undefined' && host !== 'accumulo.apache.org') {
+      $('#non-canonical').show();
+    }
+  });
+
+  
+
+  $(function() {
+    // decorate section headers with anchors
+    return $("h2, h3, h4, h5, h6").each(function(i, el) {
+      var $el, icon, id;
+      $el = $(el);
+      id = $el.attr('id');
+      icon = '<i class="fa fa-link"></i>';
+      if (id) {
+        return $el.append($("<a />").addClass("header-link").attr("href", "#" + id).html(icon));
+      }
+    });
+  });
+
+  // configure Google Analytics
+  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+  })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+  if (ga.hasOwnProperty('loaded') && ga.loaded === true) {
+    ga('create', 'UA-50934829-1', 'apache.org');
+    ga('send', 'pageview');
+  }
+</script>
+</head>
+<body style="padding-top: 100px">
+
+  <nav class="navbar navbar-default navbar-fixed-top">
+  <div class="container">
+    <div class="navbar-header">
+      <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-items">
+        <span class="sr-only">Toggle navigation</span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+      </button>
+      <a href="/"><img id="nav-logo" alt="Apache Accumulo" class="img-responsive" src="/images/accumulo-logo.png" width="200"/></a>
+    </div>
+    <div class="collapse navbar-collapse" id="navbar-items">
+      <ul class="nav navbar-nav">
+        <li class="nav-link"><a href="/downloads">Download</a></li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li class="dropdown-header">1.8.0 Release (Latest)</li>
+          <li id="nav_rn_180"><a href="/release_notes/1.8.0">Release Notes</a></li>
+          <li><a href="/1.8/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.8/apidocs">Javadoc</a></li>
+          <li id="nav_examples_18"><a href="/1.8/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">1.7.2 Release</li>
+          <li id="nav_rn_172"><a href="/release_notes/1.7.2">Release Notes</a></li>
+          <li><a href="/1.7/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.7/apidocs">Javadoc</a></li>
+          <li id="nav_examples_17"><a href="/1.7/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">1.6.6 Release</li>
+          <li id="nav_rn_166"><a href="/release_notes/1.6.6">Release Notes</a></li>
+          <li><a href="/1.6/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.6/apidocs">Javadoc</a></li>
+          <li id="nav_examples_16"><a href="/1.6/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li id="nav_features"><a href="/notable_features">Features</a></li>
+          <li id="nav_rn"><a href="/release_notes">Release Notes Archive</a></li>
+          <li id="old_documentation"><a href="/old_documentation">Documentation Archive</a></li>
+          <li id="nav_screenshots"><a href="/screenshots">Screenshots</a></li>
+          <li id="nav_papers"><a href="/papers">Papers &amp; Presentations</a></li>
+          <li id="nav_glossary"><a href="/glossary">Glossary</a></li>
+        </ul>
+        </li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li id="nav_getinvolved"><a href="/get_involved">Get Involved</a></li>
+          <li id="nav_mailinglists"><a href="/mailing_list">Mailing Lists</a></li>
+          <li id="nav_people"><a href="/people">People</a></li>
+          <li id="nav_news"><a href="/news/">News Archive</a></li>
+          <li id="nav_projects"><a href="/projects">Community Projects</a></li>
+          <li id="nav_thanks"><a href="/thanks">Thanks</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">Governance</li>
+          <li id="nav_bylaws"><a href="/bylaws">Bylaws</a></li>
+          <li id="nav_consensusbuilding"><a href="/governance/consensusBuilding">Consensus Building</a></li>
+          <li id="nav_lazyconsensus"><a href="/governance/lazyConsensus">Lazy Consensus</a></li>
+          <li id="nav_govreleasing"><a href="/governance/releasing">Releasing</a></li>
+          <li id="nav_voting"><a href="/governance/voting">Voting</a></li>
+        </ul>
+        </li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Development<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li><a href="https://issues.apache.org/jira/browse/ACCUMULO">Issue Tracker <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://github.com/apache/accumulo/pulls">Pull Requests <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://builds.apache.org/view/A/view/Accumulo">Jenkins Builds <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://travis-ci.org/apache/accumulo">TravisCI Builds <i class="fa fa-external-link"></i></a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">Guides</li>
+          <li id="nav_source"><a href="/source">Source &amp; Guide</a></li>
+          <li id="nav_git"><a href="/git">Git Workflow</a></li>
+          <li id="nav_versioning"><a href="/versioning">Versioning</a></li>
+          <li id="nav_contrib"><a href="/contrib">Contrib Projects</a></li>
+          <li id="nav_rb"><a href="/rb">Review Board</a></li>
+          <li id="nav_releasing"><a href="/releasing">Making Releases</a></li>
+          <li id="nav_verify_release"><a href="/verifying_releases">Verifying Releases</a></li>
+        </ul>
+        </li>
+      </ul>
+      <ul class="nav navbar-nav navbar-right">
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li><a href="https://www.apache.org">Apache Homepage <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/security">Security <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/thanks">Thanks <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct <i class="fa fa-external-link"></i></a></li>
+        </ul>
+        </li>
+      </ul>
+    </div>
+  </div>
+</nav>
+
+
+  <div class="container">
+    <div class="row">
+      <div class="col-md-12">
+
+        <div id="non-canonical" style="display: none; background-color: #F0E68C; padding-left: 1em;">
+          Visit the official site at: <a href="https://accumulo.apache.org">https://accumulo.apache.org</a>
+        </div>
+        <div id="content">
+          
+          <h1 class="title">Functional reads over Accumulo</h1>
+          
+          <h5>Josh Elser - 09 Jul 2014 </h5>
+
+<p>Originally posted at <a href="https://blogs.apache.org/accumulo/">https://blogs.apache.org/accumulo/</a></p>
+
+<p>Table structure is a common area of discussion between all types of Accumulo users. In the relational database realm, there was often a straightforward way that most users could agree upon that would be ideal to store and query some dataset. Data was identified by its schema, some fixed set of columns where each value within that column had some given characteristic. One of the big pushes behind the \u201cNoSQL\u201d movement was a growing pain in representing evolving data within a static schema. Applications like Accumulo removed that notion for a more flexible layout where the columns vary per row, but this flexibility often sparks debates about how data is \u201cbest\u201d stored that often ends without a clear-cut winner.</p>
+
+<p>In general, I\u2019ve found that, with new users to Accumulo, it\u2019s difficult to move beyond the basic concept of GETs and PUTs of some value for a key. Rightfully so, it\u2019s analogous to a spreadsheet: get or update the cell in the given row and column. However, there\u2019s a big difference in that the spreadsheet is running on your local desktop, instead of running across many machines. In the same way, while a local spreadsheet application has some similar functionality to Accumulo, it doesn\u2019t really make sense to think about using Accumulo as you would a spreadsheet application. Personally, I\u2019ve developed a functional-programming-inspired model which I tend to follow when implementing applications against Accumulo. The model encourages simple, efficient and easily testable code, mainly as a product of modeling the client interactions against Accumulo\u2019s APIs.</p>
+
+<h3 id="read-apis">Read APIs</h3>
+
+<p>Accumulo has two main classes for reading data from an Accumulo table: the Scanner and BatchScanner. Both accept Range(s) which limit the data read from the table based on a start and stop Key. Only data from the table that falls within those start and stop keys will be returned to the client. The reason that we have two \u201ctypes\u201d of classes to read data is that a Scanner will return data from a single Range in sorted order whereas the BatchScanner accepts multiple Ranges and returns the data unordered. In terms of Java language specifics, both the Scanner and BatchScanner are also Iterables, which return a Java Iterator that can be easily passed to some other function, transformation or for-loop.</p>
+
+<p>Having both a sorted, synchronous stream and an unsorted stream of Key-Value pairs from many servers in parallel allows for a variety of algorithms to be implemented against Accumulo. Both constructs allow for the transparency in where the data came from and encourage light-weight processing of those results on the client.</p>
+
+<h3 id="accumulo-iterators">Accumulo Iterators</h3>
+
+<p>One notable feature of Accumulo is the SortedKeyValueIterator interface, or, more succinctly, Accumulo Iterators. Typically, these iterators run inside of the TabletServer process and perform much of the heavy lifting. Iterators are used to implement a breadth of internal features such as merged file reads, visibility label filtering, versioning, and more. However, users also have the ability to leverage this server-side processing mechanism to deploy their own custom code.</p>
+
+<p>One interesting detail about these iterators is that they each have an implicit source which provides them data to operate on. This source is also a SortedKeyValueIterator which means that the \u201clocal\u201d SortedKeyValueIterator can use its own API on its data source. With this implicit hierarchy, Iterators act in concert with each other in some fixed order - they are stackable. The order in which Iterators are constructed, controlled by an Iterator\u2019s priority, determines the order of the stack. An Iterator uses its \u201csource\u201d Iterator to read data, performs some operation, and then passes it on (the next element could be a client or another Iterator). The design behind iterators deserves its own blog post; however, the concept to see here is that iterators are best designed as stateless as possible (transformations, filters, or aggregations that always net the same results given the same input).</p>
+
+<h3 id="functional-influences">Functional Influences</h3>
+
+<p>In practice, these two concepts mesh very well with each other. Data read from a table can be thought of as a \u201cstream\u201d which came from some number of operations on the server. For a Scanner, this stream of data is backed by one tablet at a time to preserve sorted-order of the table. In the case of the BatchScanner, this is happening in parallel across many tablets from many tabletservers, with the client receiving data from many distinct hosts at one time. Likewise, the Scanner and BatchScanner APIs also encourage stateless processing of this data by presenting the data as a Java Iterator. Exposing explicit batches of Key-Value pairs would encourage blocking processing of each batch would be counter-intuitive to what the server-side processing model is. It creates a more seamless implementation paradigm on both the client and the server.</p>
+
+<p>When we take a step back from Object-Oriented Java and start to think about applications in a Functional mindset, it becomes clear how these APIs encourage functional-esque code. We are less concerned about mutability and encapsulation, and more concerned about stateless operations over some immutable data. Modeling our client code like this helps encourage parallelism as application in some multi-threaded environment is much simpler.</p>
+
+<h3 id="practical-application">Practical Application</h3>
+
+<p>I started out talking about schemas and table layouts which might seem a bit unrelated to this discussion on the functional influences in the Accumulo API. Any decisions made on a table structure must take query requirements with respect to the underlying data into account. As a practical application of what might otherwise seem like pontification, let\u2019s consider a hypothetical system that processes clickstream data using Accumulo.</p>
+
+<p>Clickstream data refers to logging users who visit a website, typically for the purpose of understanding usage patterns. If a website is thought of as a directed graph, where an anchor on one page which links to another page is an edge in that graph, a user\u2019s actions on that website can be thought of as a \u201cwalk\u201d over that graph. In managing a website, it\u2019s typically very useful to understand usage patterns of your site: what page is most common? which links are most commonly clicked? what changes to a page make users act differently?</p>
+
+<p>Now, let\u2019s abstractly consider that we store this clickstream data in Accumulo. Let\u2019s not go into specifics, but say we retain the typical row-with-columns idea: each row represents some user visiting a page on your website using a globally unique identifier. Each column would contain some information about that visit: the user who is visiting the website, the page they\u2019re visiting, the page they came from, the web-browser user-agent string, etc. Say you\u2019re the owner of this website, and you recently made a modification to you website which added a prominent link to some new content on the front-page. You want to know how many people are visiting your new content with this new link you\u2019ve added, so we want to answer the question \u201chow many times was our new link on the index page clicked by any user?\u201d. For the purposes of this example, let\u2019s assume we don\u2019t have any index tables which might help us answer this query more efficiently.</p>
+
+<p>Let\u2019s think about this query in terms of stateless operations and performing as much of a reduction in data returned to the client as possible. We have a few basic steps:</p>
+
+<ol>
+  <li>Filter: Ignore all clickstream events that are not for the index page.</li>
+  <li>Filter: Ignore all clickstream events that are not for the given anchor.</li>
+  <li>Aggregation: Only a sum of the occurrences is needed, not the full record.</li>
+</ol>
+
+<p>The beauty in using Accumulo is that all three of these operations can be performed inside of the tablet server process without returning any unnecessary data to the client. Unwanted records can be easily skipped, while each record that matches our criteria is reduced to a single \u201c+1\u201d counter. Instead of returning each full record to the client, the tablet server can combine these counts together and simply return a sum to the client for the Range of data that was read.</p>
+
+<p>The other perk of thinking about the larger problem in discrete steps is that it is easily parallelized. Assuming we have many tablet servers hosting the tablets which make up our clickstream table, we can easily run this query in parallel across them all using the BatchScanner. Additionally, because we\u2019ve reduced our initial problem from a large collection of records to a stream of partial sums, we\u2019ve drastically reduced the amount of work that must be performed on our (single) client. Each key-value pair returned by a server is a partial-sum which can be combined together in a very lightweight operation (in both memory and computation) as the result is made available. The client then has the simple task of performing one summation. We took a hard problem and performed an extreme amount of heavy lifting server-side while performing next-to-no computation in our client which is great for web applications or thin clients.</p>
+
+<h3 id="tiered-computation">Tiered Computation</h3>
+
+<p>This type of algorithm, a multi-stage computation, becomes very common when working with Accumulo because of the ability to push large amounts of computation to each tablet server. Tablet servers can compute aggregations, filters and/or transformations very \u201cclose\u201d to the actual data, return some reduced view of the data being read. Even when some function is very efficient, computing it over large data sets can still be extremely time-consuming. Eliminating unwanted data as early as possible can often outweigh even the most optimal algorithms due to the orders of magnitude difference in the speed of CPU over disk and network.</p>
+
+<p>It\u2019s important to remember that this idea isn\u2019t new, though. The above model is actually very reminiscent of the MapReduce paradigm, just applied with different constraints. The types of problems efficiently solvable by MapReduce is also a super-set of what is possible with one representation of data stored in Accumulo. This also isn\u2019t a recommendation Accumulo Iterators are not a complete replacement for MapReduce (a tool is rarely a 100% \u201cbetter\u201d replacement for another). In fact, Accumulo Iterators are often used as another level of computation to make an existing MapReduce job more efficient, typically through the AccumuloInputFormat.</p>
+
+<p>We\u2019ve identified a category of problems - a function is applied to a batch of key-value pairs which reduces the complexity of a question asked over a distributed dataset - in which the features and APIs of Accumulo lend themselves extremely well to solving in an efficient and simple manner. The ability to leverage Accumulo to perform these computations requires foresight into the types of questions that are to be asked of a dataset, the structure of the dataset within Accumulo, and the reduction of a larger problem into discrete functions which are each applied to the dataset by an Accumulo Iterator.</p>
+
+
+
+<p><strong>View all posts in the <a href="/news/">news archive</a></strong></p>
+
+        </div>
+
+        
+<footer>
+
+  <p><a href="https://www.apache.org"><img src="/images/feather-small.gif" alt="Apache Software Foundation" id="asf-logo" height="100" /></a></p>
+
+  <p>Copyright � 2011-2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache�License,�Version�2.0</a>.</p>
+
+</footer>
+
+
+      </div>
+    </div>
+  </div>
+</body>
+</html>

http://git-wip-us.apache.org/repos/asf/accumulo/blob/56754f27/blog/2014/09/02/generating-keystores-for-configuring-accumulo-with-ssl.html
----------------------------------------------------------------------
diff --git a/blog/2014/09/02/generating-keystores-for-configuring-accumulo-with-ssl.html b/blog/2014/09/02/generating-keystores-for-configuring-accumulo-with-ssl.html
new file mode 100644
index 0000000..db1f502
--- /dev/null
+++ b/blog/2014/09/02/generating-keystores-for-configuring-accumulo-with-ssl.html
@@ -0,0 +1,309 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+    Licensed to the Apache Software Foundation (ASF) under one or more
+    contributor license agreements.  See the NOTICE file distributed with
+    this work for additional information regarding copyright ownership.
+    The ASF licenses this file to You under the Apache License, Version 2.0
+    (the "License"); you may not use this file except in compliance with
+    the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+-->
+<meta charset="utf-8">
+<meta http-equiv="X-UA-Compatible" content="IE=edge">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<link href="/css/bootstrap.min.css" rel="stylesheet">
+<link href="/css/accumulo.css" rel="stylesheet" type="text/css">
+<link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
+<link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/v/bs/jqc-1.12.3/dt-1.10.12/datatables.min.css"/>
+
+<title>Generating Keystores for configuring Accumulo with SSL</title>
+
+<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+<script src="/js/bootstrap.min.js"></script>
+<script type="text/javascript" src="https://cdn.datatables.net/v/bs-3.3.6/jqc-1.12.3/dt-1.10.12/datatables.min.js"></script>
+
+<script>
+  // show location of canonical site if not currently on the canonical site
+  $(function() {
+    var host = window.location.host;
+    if (typeof host !== 'undefined' && host !== 'accumulo.apache.org') {
+      $('#non-canonical').show();
+    }
+  });
+
+  
+
+  $(function() {
+    // decorate section headers with anchors
+    return $("h2, h3, h4, h5, h6").each(function(i, el) {
+      var $el, icon, id;
+      $el = $(el);
+      id = $el.attr('id');
+      icon = '<i class="fa fa-link"></i>';
+      if (id) {
+        return $el.append($("<a />").addClass("header-link").attr("href", "#" + id).html(icon));
+      }
+    });
+  });
+
+  // configure Google Analytics
+  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+  })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+  if (ga.hasOwnProperty('loaded') && ga.loaded === true) {
+    ga('create', 'UA-50934829-1', 'apache.org');
+    ga('send', 'pageview');
+  }
+</script>
+</head>
+<body style="padding-top: 100px">
+
+  <nav class="navbar navbar-default navbar-fixed-top">
+  <div class="container">
+    <div class="navbar-header">
+      <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-items">
+        <span class="sr-only">Toggle navigation</span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+      </button>
+      <a href="/"><img id="nav-logo" alt="Apache Accumulo" class="img-responsive" src="/images/accumulo-logo.png" width="200"/></a>
+    </div>
+    <div class="collapse navbar-collapse" id="navbar-items">
+      <ul class="nav navbar-nav">
+        <li class="nav-link"><a href="/downloads">Download</a></li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li class="dropdown-header">1.8.0 Release (Latest)</li>
+          <li id="nav_rn_180"><a href="/release_notes/1.8.0">Release Notes</a></li>
+          <li><a href="/1.8/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.8/apidocs">Javadoc</a></li>
+          <li id="nav_examples_18"><a href="/1.8/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">1.7.2 Release</li>
+          <li id="nav_rn_172"><a href="/release_notes/1.7.2">Release Notes</a></li>
+          <li><a href="/1.7/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.7/apidocs">Javadoc</a></li>
+          <li id="nav_examples_17"><a href="/1.7/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">1.6.6 Release</li>
+          <li id="nav_rn_166"><a href="/release_notes/1.6.6">Release Notes</a></li>
+          <li><a href="/1.6/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.6/apidocs">Javadoc</a></li>
+          <li id="nav_examples_16"><a href="/1.6/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li id="nav_features"><a href="/notable_features">Features</a></li>
+          <li id="nav_rn"><a href="/release_notes">Release Notes Archive</a></li>
+          <li id="old_documentation"><a href="/old_documentation">Documentation Archive</a></li>
+          <li id="nav_screenshots"><a href="/screenshots">Screenshots</a></li>
+          <li id="nav_papers"><a href="/papers">Papers &amp; Presentations</a></li>
+          <li id="nav_glossary"><a href="/glossary">Glossary</a></li>
+        </ul>
+        </li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li id="nav_getinvolved"><a href="/get_involved">Get Involved</a></li>
+          <li id="nav_mailinglists"><a href="/mailing_list">Mailing Lists</a></li>
+          <li id="nav_people"><a href="/people">People</a></li>
+          <li id="nav_news"><a href="/news/">News Archive</a></li>
+          <li id="nav_projects"><a href="/projects">Community Projects</a></li>
+          <li id="nav_thanks"><a href="/thanks">Thanks</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">Governance</li>
+          <li id="nav_bylaws"><a href="/bylaws">Bylaws</a></li>
+          <li id="nav_consensusbuilding"><a href="/governance/consensusBuilding">Consensus Building</a></li>
+          <li id="nav_lazyconsensus"><a href="/governance/lazyConsensus">Lazy Consensus</a></li>
+          <li id="nav_govreleasing"><a href="/governance/releasing">Releasing</a></li>
+          <li id="nav_voting"><a href="/governance/voting">Voting</a></li>
+        </ul>
+        </li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Development<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li><a href="https://issues.apache.org/jira/browse/ACCUMULO">Issue Tracker <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://github.com/apache/accumulo/pulls">Pull Requests <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://builds.apache.org/view/A/view/Accumulo">Jenkins Builds <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://travis-ci.org/apache/accumulo">TravisCI Builds <i class="fa fa-external-link"></i></a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">Guides</li>
+          <li id="nav_source"><a href="/source">Source &amp; Guide</a></li>
+          <li id="nav_git"><a href="/git">Git Workflow</a></li>
+          <li id="nav_versioning"><a href="/versioning">Versioning</a></li>
+          <li id="nav_contrib"><a href="/contrib">Contrib Projects</a></li>
+          <li id="nav_rb"><a href="/rb">Review Board</a></li>
+          <li id="nav_releasing"><a href="/releasing">Making Releases</a></li>
+          <li id="nav_verify_release"><a href="/verifying_releases">Verifying Releases</a></li>
+        </ul>
+        </li>
+      </ul>
+      <ul class="nav navbar-nav navbar-right">
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li><a href="https://www.apache.org">Apache Homepage <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/security">Security <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/thanks">Thanks <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct <i class="fa fa-external-link"></i></a></li>
+        </ul>
+        </li>
+      </ul>
+    </div>
+  </div>
+</nav>
+
+
+  <div class="container">
+    <div class="row">
+      <div class="col-md-12">
+
+        <div id="non-canonical" style="display: none; background-color: #F0E68C; padding-left: 1em;">
+          Visit the official site at: <a href="https://accumulo.apache.org">https://accumulo.apache.org</a>
+        </div>
+        <div id="content">
+          
+          <h1 class="title">Generating Keystores for configuring Accumulo with SSL</h1>
+          
+          <h5>Josh Elser - 02 Sep 2014 </h5>
+
+<p>Originally posted at <a href="https://blogs.apache.org/accumulo/">https://blogs.apache.org/accumulo/</a></p>
+
+<p>One of the major features added in Accumulo 1.6.0 was the ability to configure Accumulo so that the Thrift communications will run over SSL. <a href="http://thrift.apache.org/">Apache Thrift</a> is the remote procedure call library that is leverage for both intra-server communication and client communication with Accumulo. Issuing these calls over a secure socket ensures that unwanted actors cannot inspect the traffic sent across the wire. Given the sometimes sensitive nature of data stored in Accumulo and the authentication details for users, ensuring that no prying eyes have access to these communications is critical.</p>
+
+<p>Due to the complex and deployment specific nature of the security model for some system, Accumulo expects users to provide their own certificates, guaranteeing that they are, in fact, secure. However, for those who want to get security who do not already operate within the confines of an established security infrastructure, OpenSSL and the Java keytool command can be used to generate the necessary components to enable wire encryption.</p>
+
+<p>To enable SSL with Accumulo, it is necessary to generate a certificate authority and certificates which are signed by that authority. Typically, each client and server has its own certificate which provides the finest level of control over a secure cluster when the certificates are properly secured.</p>
+
+<h2 id="generate-a-certificate-authority">Generate a Certificate Authority</h2>
+
+<p>The certificate authority (CA) is what controls what certificates can be used to authenticate with each other. To create a secure connection with two certificates, each certificate must be signed by a certificate authority in the \u201ctruststore\u201d (A Java KeyStore which contains at least one Certificate Authority\u2019s public key). When creating your own certificate authority, a single CA is typically sufficient (and would result in a single public key in the truststore). Alternatively, a third party can also act as a certificate authority (to add an additional layer of security); however, these are typically not a free service.</p>
+
+<p>The below is an example of creating a certificate authority and adding its public key to a Java KeyStore to provide to Accumulo.</p>
+
+<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="c"># Create a private key</span>
+openssl genrsa -des3 -out root.key 4096
+
+<span class="c"># Create a certificate request using the private key</span>
+openssl req -x509 -new -key root.key -days 365 -out root.pem
+
+<span class="c"># Generate a Base64-encoded version of the PEM just created</span>
+openssl x509 -outform der -in root.pem -out root.der
+
+<span class="c"># Import the key into a Java KeyStore</span>
+keytool -import -alias root-key -keystore truststore.jks -file root.der
+
+<span class="c"># Remove the DER formatted key file (as we don't need it anymore)</span>
+rm root.der
+</code></pre>
+</div>
+
+<p>Remember to protect root.key and never distribute it as the private key is the basis for your circle of trust. The keytool command will prompt you about whether or not the certificate should be trusted: enter \u201cyes\u201d. The truststore.jks file, a \u201ctruststore\u201d, is meant to be shared with all parties communicating with one another. The password provided to the truststore verifies that the contents of the truststore have not been tampered with.</p>
+
+<h2 id="generate-a-certificatekeystore-per-host">Generate a certificate/keystore per host</h2>
+
+<p>For each host in the system, it\u2019s desirable to generate a certificate. Typically, this corresponds to a certificate per host. Additionally, each client connecting to the Accumulo instance running with SSL should be issued their own certificate. By issuing individual certificates to each entity, it gives proper control to revoke/reissue certificates to clients as necessary, without widespread interruption.</p>
+
+<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="c"># Create the private key for our server</span>
+openssl genrsa -out server.key 4096
+
+<span class="c"># Generate a certificate signing request (CSR) with our private key</span>
+openssl req -new -key server.key -out server.csr
+
+<span class="c"># Use the CSR and the CA to create a certificate for the server (a reply to the CSR)</span>
+openssl x509 -req -in server.csr -CA root.pem -CAkey root.key -CAcreateserial -out server.crt -days 365
+
+<span class="c"># Use the certificate and the private key for our server to create PKCS12 file</span>
+openssl pkcs12 -export -in server.crt -inkey server.key -certfile server.crt -name <span class="s1">'server-key'</span> -out server.p12
+
+<span class="c"># Create a Java KeyStore for the server using the PKCS12 file (private key)</span>
+keytool -importkeystore -srckeystore server.p12 -srcstoretype pkcs12 -destkeystore server.jks -deststoretype JKS
+
+<span class="c"># Remove the PKCS12 file as we don't need it</span>
+rm server.p12
+
+<span class="c"># Import the CA-signed certificate to the keystore</span>
+keytool -import -trustcacerts -alias server-crt -file server.crt -keystore server.jks
+</code></pre>
+</div>
+
+<p>These commands create a private key for the server, generated a certificate signing request created from that private key, used the certificate authority to generate the certificate using the signing request and then created a Java KeyStore with the certificate and the private key for our server. This, paired with the truststore, provide what is needed to configure Accumulo servers to run over SSL. Both the private key (server.key), the certificate signed by the CA (server.pem), and the keystore (server.jks) should be restricted to only be accessed by the user running Accumulo on the host it was generated for. Use chown and chmod to protect the files and do not distribute them over insecure networks.</p>
+
+<h2 id="configure-accumulo-servers">Configure Accumulo Servers</h2>
+
+<p>Now that the Java KeyStores have been created with the necessary information, the Accumulo configuration must be updated so that Accumulo creates the Thrift server over SSL instead of a normal socket. In accumulo-site.xml, configure the following:</p>
+
+<div class="language-xml highlighter-rouge"><pre class="highlight"><code><span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>rpc.javax.net.ssl.keyStore<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>/path/to/server.jks<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>rpc.javax.net.ssl.keyStorePassword<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>server_password<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>rpc.javax.net.ssl.trustStore<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>/path/to/truststore.jks<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>rpc.javax.net.ssl.trustStorePassword<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>truststore_password<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+<span class="nt">&lt;property&gt;</span>
+  <span class="nt">&lt;name&gt;</span>instance.rpc.ssl.enabled<span class="nt">&lt;/name&gt;</span>
+  <span class="nt">&lt;value&gt;</span>true<span class="nt">&lt;/value&gt;</span>
+<span class="nt">&lt;/property&gt;</span>
+</code></pre>
+</div>
+
+<p>The keystore and truststore paths are both absolute paths on the local filesystem (not HDFS). Remember that the server keystore should only be readable by the user running Accumulo and, if you place plaintext passwords in accumulo-site.xml, make sure that accumulo-site.xml is also not globally readable. To keep these passwords out of accumulo-site.xml, consider configuring your system with the new Hadoop CredentialProvider class, see <a href="https://issues.apache.org/jira/browse/ACCUMULO-2464">ACCUMULO-2464</a> for more information which will be available in Accumulo-1.6.1.</p>
+
+<p>Also, be aware that if unique passwords are used for each server when generating the certificate, this will result in different accumulo-site.xml files for each host. Unique configuration files per host will add much complexity to the configuration management of your instance. The use of a CredentialProvider, a feature from Hadoop which allows for acquisitions of passwords from alternate systems) can be used to help alleviate the unique accumulo-site.xml files on each host. A Java KeyStore can be created using the CredentialProvider tools which removes the necessity of passwords to be stored in accumulo-site.xml and can instead point to the CredentialProvider URI which is consistent across hosts.</p>
+
+<h2 id="configure-accumulo-clients">Configure Accumulo Clients</h2>
+
+<p>To configure Accumulo clients, use $HOME/.accumulo/config. This is a simple <a href="http://en.wikipedia.org/wiki/.properties">Java properties file</a>: each line is a configuration, key and value can be separated by a space, and lines beginning with a # symbol are ignored. For example, if we generated a certificate and placed it in a keystore (as described above), we would generate the following file for the Accumulo client.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>instance.rpc.ssl.enabled true
+rpc.javax.net.ssl.keyStore  /path/to/client-keystore.jks
+rpc.javax.net.ssl.keyStorePassword  client-password
+rpc.javax.net.ssl.trustStore  /path/to/truststore.jks
+rpc.javax.net.ssl.trustStorePassword  truststore-password
+</code></pre>
+</div>
+<p>When creating a ZooKeeperInstance, the implementation will automatically look for this file and set up a connection with the methods defined in this configuration file. The ClientConfiguration class also contains methods that can be used instead of a configuration file on the filesystem. Again, the paths to the keystore and truststore are on the local filesystem, not HDFS.</p>
+
+
+
+<p><strong>View all posts in the <a href="/news/">news archive</a></strong></p>
+
+        </div>
+
+        
+<footer>
+
+  <p><a href="https://www.apache.org"><img src="/images/feather-small.gif" alt="Apache Software Foundation" id="asf-logo" height="100" /></a></p>
+
+  <p>Copyright � 2011-2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache�License,�Version�2.0</a>.</p>
+
+</footer>
+
+
+      </div>
+    </div>
+  </div>
+</body>
+</html>

http://git-wip-us.apache.org/repos/asf/accumulo/blob/56754f27/blog/2015/03/20/balancing-groups-of-tablets.html
----------------------------------------------------------------------
diff --git a/blog/2015/03/20/balancing-groups-of-tablets.html b/blog/2015/03/20/balancing-groups-of-tablets.html
new file mode 100644
index 0000000..1349f3d
--- /dev/null
+++ b/blog/2015/03/20/balancing-groups-of-tablets.html
@@ -0,0 +1,225 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+    Licensed to the Apache Software Foundation (ASF) under one or more
+    contributor license agreements.  See the NOTICE file distributed with
+    this work for additional information regarding copyright ownership.
+    The ASF licenses this file to You under the Apache License, Version 2.0
+    (the "License"); you may not use this file except in compliance with
+    the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+-->
+<meta charset="utf-8">
+<meta http-equiv="X-UA-Compatible" content="IE=edge">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<link href="/css/bootstrap.min.css" rel="stylesheet">
+<link href="/css/accumulo.css" rel="stylesheet" type="text/css">
+<link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
+<link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/v/bs/jqc-1.12.3/dt-1.10.12/datatables.min.css"/>
+
+<title>Balancing Groups of Tablets</title>
+
+<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+<script src="/js/bootstrap.min.js"></script>
+<script type="text/javascript" src="https://cdn.datatables.net/v/bs-3.3.6/jqc-1.12.3/dt-1.10.12/datatables.min.js"></script>
+
+<script>
+  // show location of canonical site if not currently on the canonical site
+  $(function() {
+    var host = window.location.host;
+    if (typeof host !== 'undefined' && host !== 'accumulo.apache.org') {
+      $('#non-canonical').show();
+    }
+  });
+
+  
+
+  $(function() {
+    // decorate section headers with anchors
+    return $("h2, h3, h4, h5, h6").each(function(i, el) {
+      var $el, icon, id;
+      $el = $(el);
+      id = $el.attr('id');
+      icon = '<i class="fa fa-link"></i>';
+      if (id) {
+        return $el.append($("<a />").addClass("header-link").attr("href", "#" + id).html(icon));
+      }
+    });
+  });
+
+  // configure Google Analytics
+  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+  })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+  if (ga.hasOwnProperty('loaded') && ga.loaded === true) {
+    ga('create', 'UA-50934829-1', 'apache.org');
+    ga('send', 'pageview');
+  }
+</script>
+</head>
+<body style="padding-top: 100px">
+
+  <nav class="navbar navbar-default navbar-fixed-top">
+  <div class="container">
+    <div class="navbar-header">
+      <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-items">
+        <span class="sr-only">Toggle navigation</span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+      </button>
+      <a href="/"><img id="nav-logo" alt="Apache Accumulo" class="img-responsive" src="/images/accumulo-logo.png" width="200"/></a>
+    </div>
+    <div class="collapse navbar-collapse" id="navbar-items">
+      <ul class="nav navbar-nav">
+        <li class="nav-link"><a href="/downloads">Download</a></li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li class="dropdown-header">1.8.0 Release (Latest)</li>
+          <li id="nav_rn_180"><a href="/release_notes/1.8.0">Release Notes</a></li>
+          <li><a href="/1.8/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.8/apidocs">Javadoc</a></li>
+          <li id="nav_examples_18"><a href="/1.8/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">1.7.2 Release</li>
+          <li id="nav_rn_172"><a href="/release_notes/1.7.2">Release Notes</a></li>
+          <li><a href="/1.7/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.7/apidocs">Javadoc</a></li>
+          <li id="nav_examples_17"><a href="/1.7/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">1.6.6 Release</li>
+          <li id="nav_rn_166"><a href="/release_notes/1.6.6">Release Notes</a></li>
+          <li><a href="/1.6/accumulo_user_manual">User Manual</a></li>
+          <li><a href="/1.6/apidocs">Javadoc</a></li>
+          <li id="nav_examples_16"><a href="/1.6/examples">Examples</a></li>
+          <li class="divider"></li>
+          <li id="nav_features"><a href="/notable_features">Features</a></li>
+          <li id="nav_rn"><a href="/release_notes">Release Notes Archive</a></li>
+          <li id="old_documentation"><a href="/old_documentation">Documentation Archive</a></li>
+          <li id="nav_screenshots"><a href="/screenshots">Screenshots</a></li>
+          <li id="nav_papers"><a href="/papers">Papers &amp; Presentations</a></li>
+          <li id="nav_glossary"><a href="/glossary">Glossary</a></li>
+        </ul>
+        </li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li id="nav_getinvolved"><a href="/get_involved">Get Involved</a></li>
+          <li id="nav_mailinglists"><a href="/mailing_list">Mailing Lists</a></li>
+          <li id="nav_people"><a href="/people">People</a></li>
+          <li id="nav_news"><a href="/news/">News Archive</a></li>
+          <li id="nav_projects"><a href="/projects">Community Projects</a></li>
+          <li id="nav_thanks"><a href="/thanks">Thanks</a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">Governance</li>
+          <li id="nav_bylaws"><a href="/bylaws">Bylaws</a></li>
+          <li id="nav_consensusbuilding"><a href="/governance/consensusBuilding">Consensus Building</a></li>
+          <li id="nav_lazyconsensus"><a href="/governance/lazyConsensus">Lazy Consensus</a></li>
+          <li id="nav_govreleasing"><a href="/governance/releasing">Releasing</a></li>
+          <li id="nav_voting"><a href="/governance/voting">Voting</a></li>
+        </ul>
+        </li>
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Development<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li><a href="https://issues.apache.org/jira/browse/ACCUMULO">Issue Tracker <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://github.com/apache/accumulo/pulls">Pull Requests <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://builds.apache.org/view/A/view/Accumulo">Jenkins Builds <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://travis-ci.org/apache/accumulo">TravisCI Builds <i class="fa fa-external-link"></i></a></li>
+          <li class="divider"></li>
+          <li class="dropdown-header">Guides</li>
+          <li id="nav_source"><a href="/source">Source &amp; Guide</a></li>
+          <li id="nav_git"><a href="/git">Git Workflow</a></li>
+          <li id="nav_versioning"><a href="/versioning">Versioning</a></li>
+          <li id="nav_contrib"><a href="/contrib">Contrib Projects</a></li>
+          <li id="nav_rb"><a href="/rb">Review Board</a></li>
+          <li id="nav_releasing"><a href="/releasing">Making Releases</a></li>
+          <li id="nav_verify_release"><a href="/verifying_releases">Verifying Releases</a></li>
+        </ul>
+        </li>
+      </ul>
+      <ul class="nav navbar-nav navbar-right">
+        <li class="dropdown">
+        <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+        <ul class="dropdown-menu">
+          <li><a href="https://www.apache.org">Apache Homepage <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/security">Security <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/thanks">Thanks <i class="fa fa-external-link"></i></a></li>
+          <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct <i class="fa fa-external-link"></i></a></li>
+        </ul>
+        </li>
+      </ul>
+    </div>
+  </div>
+</nav>
+
+
+  <div class="container">
+    <div class="row">
+      <div class="col-md-12">
+
+        <div id="non-canonical" style="display: none; background-color: #F0E68C; padding-left: 1em;">
+          Visit the official site at: <a href="https://accumulo.apache.org">https://accumulo.apache.org</a>
+        </div>
+        <div id="content">
+          
+          <h1 class="title">Balancing Groups of Tablets</h1>
+          
+          <h5>Keith Turner - 20 Mar 2015 </h5>
+
+<p>Originally posted at <a href="https://blogs.apache.org/accumulo/">https://blogs.apache.org/accumulo/</a></p>
+
+<p>Accumulo has a pluggable tablet balancer that decides where tablets should be placed. Accumulo\u2019s default configuration spreads each tables tablets evenly and randomly across the tablet servers. Each table can configure a custom balancer that does something different.</p>
+
+<p>For some applications to perform optimally, sub-ranges of a table need to be spread evenly across the cluster. Over the years I have run into multiple use cases for this situation. The latest use case was <a href="https://github.com/fluo-io/fluo/issues/361">bad performance</a> on the <a href="http://fluo.io/">Fluo</a> <a href="https://github.com/fluo-io/fluo-stress">Stress Test</a>. This test stores a tree in an Accumulo table and creates multiple tablets for each level in the tree. In parallel, the test reads data from one level and writes it up to the next level. Figure 1 below shows an example of tablet servers hosting tablets for different levels of the tree. Under this scenario if many threads are reading data from level 2 and writing up to level 1, only Tserver 1 and Tserver 2 will be utilized. So in this scenario 50% of the tablet servers are idle.</p>
+
+<p><img src="/images/blog/201503_balancer/figure1.png" alt="figure1" />
+<em>Figure 1</em></p>
+
+<p>[ACCUMULO-3439][accumulo-3949] remedied this situation with the introduction of the <a href="https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=blob;f=server/base/src/main/java/org/apache/accumulo/server/master/balancer/GroupBalancer.java;hb=b0815affade66ab04ca27b6fc3abaac400097469">GroupBalancer</a> and <a href="https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=blob;f=server/base/src/main/java/org/apache/accumulo/server/master/balancer/RegexGroupBalancer.java;hb=51fbfaf0a52dc89e8294c86c30164fb94c9f644c">RegexGroupBalancer</a> which will be available in Accumulo 1.7.0. These balancers allow a user to arbitrarily group tablets. Each group defined by the user will be evenly spread across the tablet servers. Also, the total number of groups on each tablet server is minimized. As tablets are added or removed from the table, the balancer will migrate tablets to satisfy these goals.  Much of the complexity in the GroupBalancer code comes from trying to minimize the number 
 of migrations needed to reach a good state.</p>
+
+<p>A GroupBalancer could be configured for the table in figure 1 in such a way that it grouped tablets by level. If this were done, the result may look like Figure 2 below. With this tablet to tablet server mapping, many threads reading from level 2 and writing data up to level 1 would utilize all of the tablet servers yielding better performance.</p>
+
+<p><img src="/images/blog/201503_balancer/figure2.png" alt="figure2" />
+<em>Figure 2</em></p>
+
+<p><a href="https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=blob;f=docs/src/main/resources/examples/README.rgbalancer;hb=51fbfaf0a52dc89e8294c86c30164fb94c9f644c">README.rgbalancer</a> provides a good example of configuring and using the RegexGroupBalancer. If a regular expression can not accomplish the needed grouping, then a grouping function can be written in Java. Extend GroupBalancer to write a grouping function in java. RegexGroupBalancer provides a good example of how to do this.</p>
+
+<p>When using a GroupBalancer, how Accumulo automatically splits tablets must be kept in mind. When Accumulo decides to split a tablet, it chooses the shortest possible row prefix from the tablet data that yields a good split point. Therefore its possible that a split point that is shorter than what is expected by a GroupBalancer could be chosen. The best way to avoid this situation is to pre-split the table such that it precludes this possibility.</p>
+
+<p>The Fluo Stress test is a very abstract use case. A more concrete use case for the group balancer would be using it to ensure tablets storing geographic data were spread out evenly. For example consider <a href="https://ngageoint.github.io/geowave/">GeoWave\u2019s</a> Accumulo <a href="http://ngageoint.github.io/geowave/documentation.html#architecture-accumulo">Persistence Model</a>. Tablets could be balanced such that bins related to different regions are spread out evenly. For example tablets related to each continent could be assigned a group ensuring data related to each continent is evenly spread across the cluster. Alternatively, each Tier could spread evenly across the cluster.</p>
+
+
+
+<p><strong>View all posts in the <a href="/news/">news archive</a></strong></p>
+
+        </div>
+
+        
+<footer>
+
+  <p><a href="https://www.apache.org"><img src="/images/feather-small.gif" alt="Apache Software Foundation" id="asf-logo" height="100" /></a></p>
+
+  <p>Copyright � 2011-2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache�License,�Version�2.0</a>.</p>
+
+</footer>
+
+
+      </div>
+    </div>
+  </div>
+</body>
+</html>