You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by bu...@apache.org on 2014/09/19 22:44:10 UTC
svn commit: r922881 - in /websites/staging/accumulo/trunk/content: ./
release_notes/1.5.2.html
Author: buildbot
Date: Fri Sep 19 20:44:10 2014
New Revision: 922881
Log:
Staging update by buildbot for accumulo
Modified:
websites/staging/accumulo/trunk/content/ (props changed)
websites/staging/accumulo/trunk/content/release_notes/1.5.2.html
Propchange: websites/staging/accumulo/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Fri Sep 19 20:44:10 2014
@@ -1 +1 @@
-1626319
+1626327
Modified: websites/staging/accumulo/trunk/content/release_notes/1.5.2.html
==============================================================================
--- websites/staging/accumulo/trunk/content/release_notes/1.5.2.html (original)
+++ websites/staging/accumulo/trunk/content/release_notes/1.5.2.html Fri Sep 19 20:44:10 2014
@@ -34,7 +34,7 @@
<script src="/js/dataTables.bootstrap.js"></script>
<link href="/css/accumulo.css" rel="stylesheet" type="text/css">
<link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
- <title>Apache Accumulo 1.5.1 Release Notes</title>
+ <title>Apache Accumulo 1.5.2 Release Notes</title>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
@@ -190,127 +190,69 @@ Latest 1.5 release: <strong>1.5.1</stron
<div id="content">
- <h1 class="title">Apache Accumulo 1.5.1 Release Notes</h1>
+ <h1 class="title">Apache Accumulo 1.5.2 Release Notes</h1>
- <p>Apache Accumulo 1.5.1 is a maintenance release on the 1.5 version branch.
-This release contains changes from over 200 issues, comprised of bug fixes
+ <p>Apache Accumulo 1.5.2 is a maintenance release on the 1.5 version branch.
+This release contains changes from over 100 issues, comprised of bug fixes
(client side and server side), new test cases, and updated Hadoop support
contributed by over 30 different contributors and committers.
-As this is a maintenance release, Apache Accumulo 1.5.1 has no client API
-incompatibilities over Apache Accumulo 1.5.0 and requires no manual upgrade
-process. Users of 1.5.0 are strongly encouraged to update as soon as possible
+As this is a maintenance release, Apache Accumulo 1.5.2 has no client API
+incompatibilities over Apache Accumulo 1.5.0 and 1.5.1 and requires no manual upgrade
+process. Users of 1.5.0 or 1.5.1 are strongly encouraged to update as soon as possible
to benefit from the improvements.</p>
+<p>Users who are new to Accumulo are encouraged to use a 1.6 release as opposed
+to the 1.5 line as development has already shifted towards the 1.6 line. For those
+who cannot or do not want to upgrade to 1.6, 1.5.2 is still an excellent choice
+over earlier versions in the 1.5 line.</p>
<h2 id="notable-improvements">Notable Improvements</h2>
-<p>While new features are typically not added in a bug-fix release as 1.5.1, the
+<p>While new features are typically not added in a bug-fix release as 1.5.2, the
community does create a variety of improvements that are API compatible. Contained
here are some of the more notable improvements.</p>
-<h3 id="permgen-leak-from-client-api">PermGen Leak from Client API</h3>
-<p>Accumulo's client code creates background threads that users presently cannot
-stop through the API. This is quick to cause problems when invoking the Accumulo
-API in application containers such as Apache Tomcat or JBoss and repeatedly
-redeploying an application. <a href="https://issues.apache.org/jira/browse/ACCUMULO-2128">ACCUMULO-2128</a> introduces a static utility,
-org.apache.accumulo.core.util.CleanUp, that users can invoke as part of a
-teardown hook in their container that will stop these threads and avoid
-the eventual OutOfMemoryError "PermGen space".</p>
-<h3 id="prefer-ipv4-when-starting-accumulo-processes">Prefer IPv4 when starting Accumulo processes</h3>
-<p>While Hadoop <a href="http://wiki.apache.org/hadoop/HadoopIPv6">does not support IPv6 networks</a>, attempting to run on a
-system that does not have IPv6 completely disabled can cause strange failures.
-<a href="https://issues.apache.org/jira/browse/ACCUMULO-2262">ACCUMULO-2262</a> invokes the JVM-provided configuration parameter at process
-startup to prefer IPv4 over IPv6.</p>
-<h3 id="memory-units-in-configuration">Memory units in configuration</h3>
-<p>In previous versions, units of memory had to be provided as upper-case (e.g. '2G', not '2g').
-Additionally, a non-intuitive error was printed when a lower-case unit was provided.
-<a href="https://issues.apache.org/jira/browse/ACCUMULO-1933">ACCUMULO-1933</a> allows lower-case memory units in all Accumulo configurations.</p>
-<h3 id="thrift-maximum-frame-size">Thrift maximum frame size</h3>
-<p>Apache Thrift is used as the internal RPC service. <a href="https://issues.apache.org/jira/browse/ACCUMULO-2360">ACCUMULO-2360</a> allows
-users to configure the maximum frame size an Accumulo server will read. This
-prevents non Accumulo client from connecting and causing memory exhaustion.</p>
-<h3 id="multitablebatchwriter-concurrency">MultiTableBatchWriter concurrency</h3>
-<p>The MultiTableBatchWriter is a class which allows multiple tables to be written to
-from a single object that maintains a single buffer for caching Mutations across all tables. This is desirable
-as it greatly simplifies the JVM heap usage from caching Mutations across
-many tables. Sadly, in Apache Accumulo 1.5.0, concurrent access to a single MultiTableBatchWriter
-heavily suffered from synchronization issues. <a href="https://issues.apache.org/jira/browse/ACCUMULO-1833">ACCUMULO-1833</a> introduces a fix
-which alleviates the blocking and idle-wait that previously occurred when multiple threads accessed
-a single MultiTableBatchWriter instance concurrently.</p>
-<h3 id="hadoop-versions">Hadoop Versions</h3>
-<p>Since Apache Accumulo 1.5.0 was released, Apache Hadoop 2.2.0 was also released
-as the first generally available (GA) Hadoop 2 release. This was a very exciting release
-for a number of reasons, but this also caused additional effort on Accumulo's part to
-ensure that Apache Accumulo continues to work across multiple Hadoop versions. Apache Accumulo 1.5.1
-should function with any recent Hadoop 1 or Hadoop 2 without any special steps, tricks or instructions
-required.</p>
+<h3 id="performance-improvements">Performance improvements</h3>
+<p>The Write-Ahead Log (WAL) files are used to ensure durability of updates made to Accumulo.
+A "sync" is called on the file in HDFS to make sure that the changes to the WAL are persisted
+to disk, which allows Accumulo to recover in the case of failure. <a href="https://issues.apache.org/jira/browse/ACCUMULO-2766">ACCUMULO-2766</a> fixed
+an issue where an operation against a WAL would unnecessarily wait for multiple syncs, slowing
+down the ingest on the system.</p>
<h2 id="notable-bug-fixes">Notable Bug Fixes</h2>
-<p>As with any Apache Accumulo release, we have numerous bug fixes that have been fixed. Most
-are very subtle and won't affect the common user; however, some notable bugs were resolved
-as a part of 1.5.1 that are rather common.</p>
-<h3 id="failure-of-zookeeper-server-in-quorum-kills-connected-accumulo-services">Failure of ZooKeeper server in quorum kills connected Accumulo services</h3>
-<p>Apache ZooKeeper provides a number of wonderful features that Accumulo uses to accomplish
-a variety of tasks, most notably a distributed locking service. Typically, multiple ZooKeeper
-servers are run to provide resilience against a certain number of node failures. <a href="https://issues.apache.org/jira/browse/ACCUMULO-1572">ACCUMULO-1572</a>
-resolves an issue where Accumulo processes would kill themselves when the ZooKeeper server they
-were communicating with died instead of failing over to another ZooKeeper server in the quorum.</p>
-<h3 id="monitor-table-state-isnt-updated">Monitor table state isn't updated</h3>
-<p>The Accumulo Monitor contains a column for the state of each table in the Accumulo instance.
-The previous resolution was to restart the Monitor process when it got in this state.
-<a href="https://issues.apache.org/jira/browse/ACCUMULO-1920">ACCUMULO-1920</a> resolves an issue where the Monitor would not see updates from ZooKeeper.</p>
-<h3 id="two-locations-for-the-same-extent">Two locations for the same extent</h3>
-<p>The !METADATA table is the brains behind the data storage for each table, tracking information
-like which files comprise a Tablet, and which TabletServers are hosting which Tablets. <a href="https://issues.apache.org/jira/browse/ACCUMULO-2057">ACCUMULO-2057</a>
-fixes an issue where the !METADATA table contained multiple locations (hosting server) for
-a single Tablet.</p>
-<h3 id="deadlock-on-metadata-tablet-unload">Deadlock on !METADATA tablet unload</h3>
-<p>Tablets are unloaded, typically, when a shutdown request is issued. <a href="https://issues.apache.org/jira/browse/ACCUMULO-1143">ACCUMULO-1143</a> resolves
-a potential deadlock issue when a merging-minor compaction is issued to flush in-memory data
-to disk before unloading a Tablet.</p>
-<h3 id="other-notable-fixes">Other notable fixes</h3>
-<ul>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1800">ACCUMULO-1800</a> Fixed deletes made via the Proxy.</li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1994">ACCUMULO-1994</a> Fixed ranges in the Proxy.</li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-2234">ACCUMULO-2234</a> Fixed offline map reduce over non default HDFS location.</li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1615">ACCUMULO-1615</a> Fixed <code>service accumulo-tserver stop</code>.</li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1876">ACCUMULO-1876</a> Fixed issues depending on Accumulo using Apache Ivy.</li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-2261">ACCUMULO-2261</a> Duplicate locations for a Tablet.</li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-2037">ACCUMULO-2037</a> Tablets assigned to previous location.</li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1821">ACCUMULO-1821</a> Avoid recovery on recovering Tablets.</li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-2078">ACCUMULO-2078</a> Incorrectly computed ACCUMULO_LOG_HOST in example configurations.</li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1985">ACCUMULO-1985</a> Configuration to bind Monitor on all network interfaces.</li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1999">ACCUMULO-1999</a> Allow '0' to signify random port for the Master.</li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1630">ACCUMULO-1630</a> Fixed GC to interpret any IP/hostname.</li>
-</ul>
-<h2 id="known-issues">Known Issues</h2>
-<p>When using Accumulo 1.5 and Hadoop 2, Accumulo will call hsync() on HDFS.
-Calling hsync improves durability by ensuring data is on disk (where other older
-Hadoop versions might lose data in the face of power failure); however, calling
-hsync frequently does noticably slow writes. A simple work around is to increase
-the value of the tserver.mutation.queue.max configuration parameter via accumulo-site.xml.</p>
-<p>A value of "4M" is a better recommendation, and memory consumption will increase by
-the number of concurrent writers to that TabletServer. For example, a value of 4M with
-50 concurrent writers would equate to approximately 200M of Java heap being used for
-mutation queues.</p>
-<p>For more information, see <a href="https://issues.apache.org/jira/browse/ACCUMULO-1950">ACCUMULO-1950</a> and <a href="https://issues.apache.org/jira/browse/ACCUMULO-1905?focusedCommentId=13915208&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13915208">this comment</a>.</p>
+<h3 id="fixes-mapreduce-package-name-change">Fixes MapReduce package name change</h3>
+<p>1.5.1 inadvertently included a change to RangeInputSplit which created an incompatibility
+with 1.5.0. The original class has been restored to ensure that users accessing
+the RangeInputSplit class do not have to alter their client code. See <a href="https://issues.apache.org/jira/browse/ACCUMULO-2586">ACCUMULO-2586</a> for
+more information</p>
+<h3 id="add-configurable-maximum-frame-size-to-thrift-proxy">Add configurable maximum frame size to Thrift proxy</h3>
+<p>The Thrift proxy server was subject to memory exhaustion, typically
+due to bad input, where the server would attempt to allocate a very large
+buffer and die in the process. <a href="https://issues.apache.org/jira/browse/ACCUMULO-2658">ACCUMULO-2658</a> introduces a configuration
+parameter, like <a href="https://issues.apache.org/jira/browse/ACCUMULO-2360">ACCUMULO-2360</a>, to prevent this error.</p>
+<h3 id="offline-tables-can-prevent-tablet-balancing">Offline tables can prevent tablet balancing</h3>
+<p>A table with many tablets was created, data ingested into it, and then taken
+offline. There were tablet migrations also queued for the table which could not
+happen because the table was offline at that point. The balancer doesn't run
+when there are outstanding migrations; therefore, the system became more and more
+out of balance. <a href="https://issues.apache.org/jira/browse/ACCUMULO-2694">ACCUMULO-2694</a> introduces a fix to ensure that offline tables
+do not block balancing and improves the server-side logging.</p>
+<h3 id="miniaccumulocluster-process-management">MiniAccumuloCluster process management</h3>
+<p>MiniAccumuloCluster had a few issues which could cause deadlock or a method that
+never returns. Most of these are related to management of the Accumulo processes
+(<a href="https://issues.apache.org/jira/browse/ACCUMULO-2764">ACCUMULO-2764</a>, <a href="https://issues.apache.org/jira/browse/ACCUMULO-2985">ACCUMULO-2985</a>, and <a href="https://issues.apache.org/jira/browse/ACCUMULO-3055">ACCUMULO-3055</a>).</p>
+<h3 id="iteratorsettings-not-correctly-serialized-in-rangeinputsplit">IteratorSettings not correctly serialized in RangeInputSplit</h3>
+<p>The Writable interface methods on the RangeInputSplit class accidentally omitted
+calls to serialize the IteratorSettings configured for the Job. <a href="https://issues.apache.org/jira/browse/ACCUMULO-2962">ACCUMULO-2962</a>
+fixes the serialization and adds some additional tests.</p>
<h2 id="documentation">Documentation</h2>
<p>The following documentation updates were made: </p>
<ul>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1956">ACCUMULO-1956</a></li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1428">ACCUMULO-1428</a></li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1687">ACCUMULO-1687</a></li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-2141">ACCUMULO-2141</a></li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1946">ACCUMULO-1946</a></li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-2223">ACCUMULO-2223</a></li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-2226">ACCUMULO-2226</a></li>
-<li><a href="https://issues.apache.org/jira/browse/ACCUMULO-1470">ACCUMULO-1470</a></li>
+<li>[ACCUMULO-1956][18]</li>
+<li>[ACCUMULO-1428][19]</li>
+<li>[ACCUMULO-1687][29]</li>
+<li>[ACCUMULO-2141][30]</li>
+<li>[ACCUMULO-1946][31]</li>
+<li>[ACCUMULO-2223][32]</li>
+<li>[ACCUMULO-2226][33]</li>
+<li>[ACCUMULO-1470][34]</li>
</ul>
<h2 id="testing">Testing</h2>
-<p>Below is a list of all platforms that 1.5.1 was tested against by developers. Each Apache Accumulo release
-has a set of tests that must be run before the candidate is capable of becoming an official release. That list includes the following:</p>
-<ol>
-<li>Successfully run all unit tests</li>
-<li>Successfully run all functional test (test/system/auto)</li>
-<li>Successfully complete two 24-hour RandomWalk tests (LongClean module), with and without "agitation"</li>
-<li>Successfully complete two 24-hour Continuous Ingest tests, with and without "agitation", with data verification</li>
-<li>Successfully complete two 72-hour Continuous Ingest tests, with and without "agitation"</li>
-</ol>
<p>Each unit and functional test only runs on a single node, while the RandomWalk and Continuous Ingest tests run
on any number of nodes. <em>Agitation</em> refers to randomly restarting Accumulo processes and Hadoop Datanode processes,
and, in HDFS High-Availability instances, forcing NameNode failover.
@@ -331,38 +273,6 @@ and, in HDFS High-Availability instances
<td>Yes (QJM)</td>
<td>All required tests</td>
</tr>
- <tr>
- <td>CentOS 6.4</td>
- <td>CDH 4.5.0 (2.0.0+cdh4.5.0)</td>
- <td>7</td>
- <td>CDH 4.5.0 (3.4.5+cdh4.5.0)</td>
- <td>Yes (QJM)</td>
- <td>Unit, functional and 24hr Randomwalk w/ agitation</td>
- </tr>
- <tr>
- <td>CentOS 6.4</td>
- <td>CDH 4.5.0 (2.0.0+cdh4.5.0)</td>
- <td>7</td>
- <td>CDH 4.5.0 (3.4.5+cdh4.5.0)</td>
- <td>Yes (QJM)</td>
- <td>2x 24/hr continuous ingest w/ verification</td>
- </tr>
- <tr>
- <td>CentOS 6.3</td>
- <td>Apache 1.0.4</td>
- <td>1</td>
- <td>Apache 3.3.5</td>
- <td>No</td>
- <td>Local testing, unit and functional tests</td>
- </tr>
- <tr>
- <td>RHEL 6.4</td>
- <td>Apache 2.2.0</td>
- <td>10</td>
- <td>Apache 3.4.5</td>
- <td>No</td>
- <td>Functional tests</td>
- </tr>
</table></p>
</div>