You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by mm...@apache.org on 2020/08/26 14:29:54 UTC
[accumulo-website] branch main updated: Initial draft of 1.10
release notes. (#218)
This is an automated email from the ASF dual-hosted git repository.
mmiller pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/main by this push:
new 22cf7d4 Initial draft of 1.10 release notes. (#218)
22cf7d4 is described below
commit 22cf7d45cfed626bfa6a6b06c1b3dbb3b097f872
Author: EdColeman <de...@etcoleman.com>
AuthorDate: Wed Aug 26 10:29:42 2020 -0400
Initial draft of 1.10 release notes. (#218)
* Initial draft of 1.10 release notes.
ToDo:
- make website / jekyill friendly.
- update inter-doc and external links.
* Update with review comments
* Update with PRs merged since February.
* Update _posts/release/2020-02-29-accumulo-1.10.0.md
Co-authored-by: Keith Turner <kt...@apache.org>
* Update _posts/release/2020-02-29-accumulo-1.10.0.md
Co-authored-by: Keith Turner <kt...@apache.org>
Co-authored-by: Ed Coleman etcoleman <ed...@apache.org>
Co-authored-by: Keith Turner <kt...@apache.org>
---
_posts/release/2020-02-29-accumulo-1.10.0.md | 216 +++++++++++++++++++++++++++
1 file changed, 216 insertions(+)
diff --git a/_posts/release/2020-02-29-accumulo-1.10.0.md b/_posts/release/2020-02-29-accumulo-1.10.0.md
new file mode 100644
index 0000000..ddff068
--- /dev/null
+++ b/_posts/release/2020-02-29-accumulo-1.10.0.md
@@ -0,0 +1,216 @@
+---
+title: Accumulo 1.10.0 (Draft)
+---
+
+28 August 2020
+
+Apache Accumulo 1.10 is a maintenance release of the 1.9.3 version with additional internal
+improvements. This release raises the Java language level for the source code
+from 1.7 to 8 and requires a Java runtime environment of Java 8 or later. The source code level
+change aligns the Accumulo code base Java language version requirements with the 2.0 branches and
+allows Java language enhancements such as lambda expressions and improved interface features to be
+used in all active branches of the Accumulo code base.
+
+With the adoption of [semver](https://semver.org/spec/v2.0.0.html), the minor number version change
+(from 1.9 to 1.10) signals this release does not modify the Accumulo public API, but that the
+release contains internal improvements beyond changes that are permitted by semver in a patch
+release. This release contains contributions from more than 13 contributors from the Apace Accumulo
+community in over 80 commits and 16 months of work since the 1.9.3 release. The following release
+notes highlight some of the changes. If anything is missing from this list, please
+[contact](https://accumulo.apache.org/contact-us) the developers to have it included.
+
+## Long Term Support (LTS)
+
+The Apache Accumulo community has adopted a formal Long Term Support (LTS) release policy. Accumulo
+version 1.10 is the first LTS version and is expected to be the last major / minor release of the
+1.x line. With the LTS designation, 1.10.0 will receive critical bug and security fixes for the
+defined support period. No new features will be added to the 1.10 line and all new feature
+development will occur on the 2.x line.
+
+| Version | Release Date | Maintenance Period |
+| :----- | :----- | :----- |
+| 1.10.x | 31 Aug 2020 | 2 years |
+| 2.1.x | 4th Quarter 2020 | 2 years |
+
+Users of 1.9.3 or earlier are urged to upgrade to 1.10.0 and to consider the starting the migration
+to prepare for the Accumulo 2.1.0 release. If you would like to start preparing for 2.1.0 now, one way to do this is to start building and testing the next version of your software against Accumulo 2.0.0.
+
+## Notable Changes
+
+### Bug Fixes
+
+#### Addressed multiple possible concurrency issues
++ Prevent multiple threads from working on same bulk file. (Issue #1153)
++ Avoid multiple threads loading same cache block (Issue #990)
++ Eliminate task creation leak caused by the addition of a new, additional timed-task for every GC
+run by creating on time-tasked instance in the GC (Issue #1314) (PR #1318)
++ Fix ConcurrentModificationException in HostRegexTableLoadBalancer (#1107)
+
+#### Improve metadata and root table processing to prevent corruption during GC.
++ Prevent cloning of the metadata table. Cloning the metadata table could result in losing
+files for either the clone, or the original metadata table during GC processing (Issue #1309)
++ Improve GC handling of WALs used by root tablet. If the root tablet had WALs, the GC did
+not consider them during collection. (PR #1310)
+
+#### Fix issue with minor compactions
+Added retry to minor compaction thread(s) to prevent transient iterator issues blocking
+forever (#1644).
+
+#### Strengthened checks for FATE RPC arguments
++ Accumulo 2.0 allows extra arguments at table creation. This fix strengths checks for FATE RPC
+arguments so that unexpected, extra arguments throw an exception instead of being silently being
+ignored if run against pre 2.0 code (Issue #1141)
+
+#### Address recovery performance issues
++ Fix WAL recovery performance issue by adding a temporarily cache for the existence of recovery
+wals For the case where a lot of tablet servers died, the master was frequently checking
+for the existence of a recovery log for each tablet. It is very likely that many tablets point to
+the same recovery logs so the existence checks are redundant. This patch caches the result of
+existence checks for a short period. (#1462)
++ Decrease recovery cache time - reduces NN exists calls (#1526)
+
+#### Fix handling of client options
++ Validate ClientOpts instance name and Zookeeper paths to fix issue where default client
+configuration was not being used. This change allows either the default configuration, or the
+command line set the instance name. (#1478)
++ Ensures correct use of ZooKeeper getAcl (#1185)
++ Expanded InputConfigurator permissions checks to include Namespace.READ (#1371)
+
+#### Fix monitor trace display
++ Fix regression where trace information could not be displayed in the accumulo monitor.(Issue #1401)
+
+### Improvements
+
+#### Improve tablet logging for hot-spot detection
++ Improve Logging of busy tablets. The deltas of tablets ingest and query counts are used for
+computing the busiest N tablets. The previous code used the absolute counts.
++ Fixed prioQ that was not properly tracking the top N tablets.(PR #1291)
++ Improve busy tracker handling of reloaded tablets (PR #1296)
+
+#### tserver start-up and shutdown protections:
++ Add option to check for a threshold number of servers to be registered and available on start-up
+before the master begins tablet assignments when master acquires lock on start-up. The options
+allow for threshold number of servers and an optional max wait period.
+See [Property Changes](#Property Change Summary) for property additions. (#1158)
++ Add throttle for the number of shutdown requests sent by the master to tservers (Issue #1456)
+
+#### Optional GC metadata operations.
++ Added a property - gc.use.full.compaction At the completion of a GC cycle, the GC compacted the
+metadata table to ensure that GC changes were flushed and persisted. The property allows for the
+action to be specified as compaction, flush or none. (#1352)
+
+#### Zookeeper 3.5 version support
++ Support ZooKeeper 3.4 and 3.5 - Update for ZK 3.5 jar location changes (#1503)
+
+#### Update log statements:
+Compacting a table without user iterators was logging like it was an abnormal condition (#1347)
+Reduce verbose logging of merge operations in Master log (#1338)
+Improve logging for session expired events. When a tserver looses lock, the Session expired message
+is a log statement for every watcher - reduce current message to trace and add summary (#1108)
+
+#### Improve shel commands command
++ Add optional -t tablename to importdirectory shell command. (#1299)
++ Fix idempotency bug in importtable (#1555)
+
+#### Add GC hadoop2 metrics.
+Add GC cycle metrics (file and wal collection) to be reported via the hadoop2 metrics. This exposes
+the gc cycle metrics available in the monitor to external metrics systems and includes run time
+for the gc post operation (compact, flush). (#1352)
++ AccGcStarted - timestamp of GC cycle start
++ AccGcFinished - timestamps of GC cycle finished
++ AccGcCandidates - Number of candidates for GC
++ AccGcInUse - Number of candidates still in use
++ AccGcDeleted - Number of candidates deleted
++ AccGcErrors - Number of deletion errors
++ AccGcWalStarted - timestamp of WAL collection start
++ AccGcWalFinished - timestamp of WAL collection completion
++ AccGcWalCandidates - number of WAL candidates for collection
++ AccGcWalInUse - number of WALs in use
++ AccGcWalDeleted - number of WALs deleted
++ AccGcWalErrors- number of errors during WAL deletion
++ AccGcPostOpDuration - duration of compact / flush
++ AccGcRunCycleCount - 1-up cycle count
+
+#### Add option allow clone table to keep the clone offline.
++ Option to leave cloned tables offline (#1475)
++ Updated the shell clone command with a -o option
+
+#### Improved metadata table consistency checking and testing during GC
+
++ During GC scans, an error will be thrown if the GC fails consistency checks (#1379)
++ Added a check to ensure the last tablet was seen. This should cause an error to be thrown if
+the scanner stops returning data for any reason.
+
+### Java Language Level Changes
+This lists some of the code changes made to update the Java language level to Java 8. They are
+listed here to aid migration of client code that may have similar issues migrating to the updated
+language level (#1467).
++ Update to maven compiler source and target to Java 1.8
++ Add optional dependency to pom for modernizer annotations
++ Add SuppressModernizer annotation to uses of Guava Predicate in AccumuloConfiguration and its
+subclasses, Optional in VolumeChooser, and Function in GroupBalancer
++ Replace Guava Function, Predicate, Preconditions, Optional and Iterators where able with java.util equivalents
++ Replace core.util.Base64 class with java.util equivalent
++ Fix warnings introduced by Java 1.8
++ Drop method in MockConfiguration causing compiler warning and made PropertyFilter in
+AccumuloConfiguration extend guava Predicate
++ Rename execute methods in MasterClient and ServerClient, like the 2.0 changes to fix the
+overload warnings
++ Add colVis file to ContinuousInputFormat. Fixes #1368 (#1369)
++ Fix unused var and generics warning
++ Limit UnsynchronizedBuffer maximum size to Integer.MAX_VALUE - 8 (#1523)
++ Remove need for ANT on classpath (#1532)
++ Update spotbugs and maven enforcer plugin versions
++ Rename yield method - future restricted identifier
++ Fix javadoc html header elements for newer JDKs
+
+### Other Changes
+
+#### Build Improvements / Changes
++ Switch to GitHub Actions and improve CI workflow (#1671)
++ Switch POM project xml attrs from HTTP to HTTPS
++ Add patch file for Java 11 generated apidocs
++ Update native map makefile to work with Java 11.
++ The makefile was modified to check if javac exists - previously it checked to see if javah exists
+inorder to determine if java is a JDK or JRE. In Java 11, javah no longer exists - changed to
+look for javac compiler command that exists in Java 8 and 11.
+
+#### Test Improvements
++ Make ITs override defaultTimeoutSeconds (#1483)
++ Increase timeouts on some slow ITs (#1482)
++ Stabilize ChaoticBalancerIT (#1491)
++ Increase timeout for GarbageCollectWALIT to allow recovery
++ Improve importDirectory tests from \#1299
++ Limit log size in Travis CI builds
+
+# New Properties
+
+Set server count threshold and maximum wait time on start-up before master begins tablet assignments:
+
+| Property | Description |
+| :----- | :----- |
+| master.startup.tserver.avail.min.count | the number of required tservers |
+| master.startup.tserver.avail.max.wait | the max time willing to wait before assigning tablets if threshold count has not been reached. |
+
+Enable improved tserver logging for hot-spot detection:
+
+| Property | Description |
+| :----- | :----- |
+| tserver.log.busy.tablets.count | the number of busiest tablets to log. Logged at interval controlled by tserver.log.busy.tablets.interval. If \<= 0, logging of busy tablets is disabled. |
+| tserver.log.busy.tablets.interval | Time interval between logging out busy tablets information. |
+
+GC changes:
+
+| Property | Description |
+| :----- | :----- |
+| gc.post.metadata.action | Allowed values: compact - default, current behavior, the metadata table is flushed and compacted on each GC run. flush - only flush the metadata table, compactions will occur according to number of files compaction rules. none - take no action. |
+| gc.metrics.enabled | Enable detailed gc metrics reporting with hadoop metrics. |
+
+# Useful links #
+
+(TODO - make actual links)
+
+Release VOTE email thread
+All changes since 1.9.3
+GitHub - List of issues tracked on GitHub corresponding to this release
+1.9.3 release notes - Release notes showing changes in the previous release