You are viewing a plain text version of this content. The canonical link for it is here.

Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2020/02/06 00:23:20 UTC

[GitHub] [accumulo-website] EdColeman opened a new pull request #218: Initial draft of 1.10 release notes.

EdColeman opened a new pull request #218: Initial draft of 1.10 release notes.
URL: https://github.com/apache/accumulo-website/pull/218
 
 
   ToDo:
    - make website / jekyill friendly.
    - update inter-doc and external links.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [accumulo-website] milleruntime commented on a change in pull request #218: Initial draft of 1.10 release notes.

Posted by GitBox <gi...@apache.org>.

milleruntime commented on a change in pull request #218: Initial draft of 1.10 release notes.
URL: https://github.com/apache/accumulo-website/pull/218#discussion_r375986251

##########
File path: _posts/release/2020-02-29-accumulo-1.10.0.md
##########
@@ -0,0 +1,191 @@
+---
+title: Accumulo 1.10.0 (Draft)
+---
+
+28 Feb 2020
+
+Apache Accumulo 1.10 is a maintenance release of the 1.9.3 version with additional internal
+improvements. This release raises the Java language level for the source code
+from 1.7 to 8 and requires a Java runtime environment of Java 8 or later. The source code level
+change aligns the Accumulo code base Java language version requirements with the 2.0 branches and
+allows Java language enhancements such as lambda expressions and improved interface features to be
+used in all active branches of the Accumulo code base.
+
+With the adoption of [semver](https://semver.org/spec/v2.0.0.html), the minor number version change
+(from 1.9 to 1.10) signals this release does not modify the Accumulo public API, but that the
+release contains internal improvements beyond changes that are permitted by semver in a patch
+release. This release contains contributions from more than 11 contributors from the Apace Accumulo
+community in over 70 commits and 10 months of work since the 1.9.3 release. The following release
+notes highlight some of the changes. If anything is missing from this list, please
+[contact](https://accumulo.apache.org/contact-us) the developers to have it included.
+
+## Long Term Support (LTS)
+
+The Apache Accumulo community has adopted a formal Long Term Support (LTS) release policy. Accumulo
+version 1.10 is the first LTS version and is expected to be the last major / minor release of the
+1.x line. With the LTS designation, 1.10.0 will receive critical bug and security fixes for the
+defined support period. No new features will be added to the 1.10 line and all new feature
+development will occur on the 2.x line.
+
+| Version | Release Date | Maintenance Period |
+| :----- | :----- | :----- |
+| 1.10.x | 28 Feb 2020 | 2 years |
+| 2.1.x | 2nd Quater 2020 | 2 years |
+
+Users of 1.9.3 or earlier are urged to upgrade to 1.10.0 and to consider the starting the migration
+to prepare for the Accumulo 2.1.0 release.
+
+## Notable Changes
+
+### Bug Fixes
+
+#### Addressed multiple possible concurrency issues
++ Prevent multiple threads from working on same bulk file. (Issue #1153)
++ Avoid multiple threads loading same cache block (Issue #990)
++ Eliminate task creation leak caused by the addition of a new, additional timed-task for every GC
+run by creating on time-tasked instance in the GC (Issue #1314) (PR #1318)
++ Fix ConcurrentModificationException in HostRegexTableLoadBalancer (#1107)
+
+#### Improve metadata and root table processing to prevent corruption during GC.
++ Prevent cloning of the metadata table. Cloning the metadata table could result in losing
+files for either the clone, or the original metadata table during GC processing (Issue #1309)
++ Improve GC handling of WALs used by root tablet. If the root tablet had WALs, the GC did
+not consider them during collection. (PR #1310)
+
+#### Strengthened checks for FATE RPC arguments
++ Accumulo 2.0 allows extra arguments at table creation. This fix strengths checks for FATE RPC
+arguments so that unexpected, extra arguments throw an exception instead of being silently being
+ignored if run against pre 2.0 code (Issue #1141)
+
+#### Fix WAL recovery performance issue
++ Fix WAL recovery performance issue by adding a temporarily cache for the existence of recovery
+wals For the case where a lot of tablet servers died, the master was frequently checking
+for the existence of a recovery log for each tablet. Its very likely that many tablets point to the
+same recovery logs and the existence checks are redundant. This patch caches the result of
+existence checks for a short period. (#1462)
+
+#### Fix handling of client options
++ Validate ClientOpts instance name and Zookeeper paths to fix issue where default client
+configuration was not being used. This change allows either the default configuration, or the
+command line set the instance name. (#1478)
++ Only override instance name and ZK paths if those were specified by caller.
++ Ensures correct use of ZooKeeper getAcl (#1185)
++ Expanded InputConfigurator permissions checks to include Namespace.READ (#1371)
+
+#### Fix monitor trace display
++ Fix regression where trace information could not be displayed in the accumulo monitor.(Issue #1401)
+
+### Improvements
+
+#### Improve tablet logging for hot-spot detection
++ Improve Logging of busy tablets. The deltas of tablets ingest and query counts are used for
+computing the busiest N tablets. The previous code used the absolute counts.
++ Fixed prioQ that was not properly tracking the top N tablets.(PR #1291)
++ Improve busy tracker handling of reloaded tablets (PR #1296)
+
+#### tserver start-up and shutdown protections:
++ Add option to check for a threshold number of servers to be registered and available on start-up
+before the master begins tablet assignments when master acquires lock on start-up. The options
+allow for threshold number of servers and an optional max wait period.
+See [Property Changes](#Property Change Summary) for property additions.
++ Add throttle for the number of shutdown requests sent by the master to tservers (Issue #1456)
+
+#### Optional GC metadata operations.
++ Added a property - gc.use.full.compaction At the completion of a GC cycle, the GC compacted the
+metadata table to ensure that GC changes were flushed and persisted. The property allows for the
+action to be specified as compaction, flush or none.

Review comment:
Missing PR number

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [accumulo-website] milleruntime commented on a change in pull request #218: Initial draft of 1.10 release notes.

Posted by GitBox <gi...@apache.org>.

milleruntime commented on a change in pull request #218: Initial draft of 1.10 release notes.
URL: https://github.com/apache/accumulo-website/pull/218#discussion_r375987293

Review comment:
Could include the PR number as well.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [accumulo-website] milleruntime commented on a change in pull request #218: Initial draft of 1.10 release notes.

Posted by GitBox <gi...@apache.org>.

milleruntime commented on a change in pull request #218: Initial draft of 1.10 release notes.
URL: https://github.com/apache/accumulo-website/pull/218#discussion_r375974800

Review comment:
I think this is the PR number: https://github.com/apache/accumulo/pull/1158

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [accumulo-website] milleruntime commented on a change in pull request #218: Initial draft of 1.10 release notes.

Posted by GitBox <gi...@apache.org>.

milleruntime commented on a change in pull request #218: Initial draft of 1.10 release notes.
URL: https://github.com/apache/accumulo-website/pull/218#discussion_r375965625

Review comment:
This line could be combined with the one above it.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [accumulo-website] milleruntime commented on a change in pull request #218: Initial draft of 1.10 release notes.

Posted by GitBox <gi...@apache.org>.

milleruntime commented on a change in pull request #218: Initial draft of 1.10 release notes.
URL: https://github.com/apache/accumulo-website/pull/218#discussion_r375988104
 
 

 ##########
 File path: _posts/release/2020-02-29-accumulo-1.10.0.md
 ##########
 @@ -0,0 +1,191 @@
+---
+title: Accumulo 1.10.0 (Draft)  
+---
+
+28 Feb 2020
+
+Apache Accumulo 1.10 is a maintenance release of the 1.9.3 version with additional internal 
+improvements.  This release raises the Java language level for the source code 
+from 1.7 to 8 and requires a Java runtime environment of Java 8 or later.  The source code level 
+change aligns the Accumulo code base Java language version requirements with the 2.0 branches and 
+allows Java language enhancements such as lambda expressions and improved interface features to be 
+used in all active branches of the Accumulo code base.
+ 
+With the adoption of [semver](https://semver.org/spec/v2.0.0.html), the minor number version change
+(from 1.9 to 1.10) signals this release does not modify the Accumulo public API, but that the 
+release contains internal improvements beyond changes that are permitted by semver in a patch 
+release. This release contains contributions from more than 11 contributors from the Apace Accumulo
+community in over 70 commits and 10 months of work since the 1.9.3 release. The following release 
+notes highlight some of the changes. If anything is missing from this list, please
+[contact](https://accumulo.apache.org/contact-us) the developers to have it included.
+
+## Long Term Support (LTS)
+
+The Apache Accumulo community has adopted a formal Long Term Support (LTS) release policy.  Accumulo 
+version 1.10 is the first LTS version and is expected to be the last major / minor release of the 
+1.x line. With the LTS designation, 1.10.0 will receive critical bug and security fixes for the 
+defined support period.  No new features will be added to the 1.10 line and all new feature 
+development will occur on the 2.x line.
+
+| Version | Release Date | Maintenance Period |
+| :----- | :----- | :----- |
+| 1.10.x | 28 Feb 2020 | 2 years |
+| 2.1.x | 2nd Quater 2020 | 2 years |
+
+Users of 1.9.3 or earlier are urged to upgrade to 1.10.0 and to consider the starting the migration 
+to prepare for the Accumulo 2.1.0 release.  
+
+## Notable Changes
+
+### Bug Fixes
+
+#### Addressed multiple possible concurrency issues
++ Prevent multiple threads from working on same bulk file. (Issue #1153)
++ Avoid multiple threads loading same cache block (Issue #990)
++ Eliminate task creation leak caused by the addition of a new, additional timed-task for every GC 
+run by creating on time-tasked instance in the GC (Issue #1314)  (PR #1318)
++ Fix ConcurrentModificationException in HostRegexTableLoadBalancer (#1107)
+
+#### Improve metadata and root table processing to prevent corruption during GC.
++ Prevent cloning of the metadata table.  Cloning the metadata table could result in losing 
+files for either the clone, or the original metadata table during GC processing (Issue #1309)
++ Improve GC handling of  WALs used by root tablet.  If the root tablet had WALs, the GC did 
+not consider them during collection.  (PR #1310)
+
+#### Strengthened checks for FATE RPC arguments
++ Accumulo 2.0 allows extra arguments at table creation. This fix strengths checks for FATE RPC 
+arguments so that unexpected, extra arguments throw an exception instead of being silently being 
+ignored if run against pre 2.0 code  (Issue #1141)
+
+#### Fix WAL recovery performance issue
++ Fix WAL recovery performance issue by adding a temporarily cache for the existence of recovery 
+wals For the case where a lot of tablet servers died, the master was frequently checking 
+for the existence of a recovery log for each tablet. Its very likely that many tablets point to the 
+same recovery logs and the existence checks are redundant.  This patch caches the result of 
+existence checks for a short period. (#1462) 
+
+#### Fix handling of client options
++ Validate ClientOpts instance name and Zookeeper paths to fix issue where default client 
+configuration was not being used. This change allows either the default configuration, or the 
+command line set the instance name. (#1478)
++ Only override instance name and ZK paths if those were specified by caller.
++ Ensures correct use of ZooKeeper getAcl (#1185)
++ Expanded InputConfigurator permissions checks to include Namespace.READ (#1371)
+
+#### Fix monitor trace display
++ Fix regression where trace information could not be displayed in the accumulo monitor.(Issue #1401)
+
+### Improvements
+
+#### Improve tablet logging for hot-spot detection
++ Improve Logging of busy tablets. The deltas of tablets ingest and query counts are used for 
+computing  the busiest N tablets. The previous code used the absolute counts.
++ Fixed prioQ that was not properly tracking the top N tablets.(PR #1291)
++ Improve busy tracker handling of reloaded tablets (PR #1296)
+    
+#### tserver start-up and shutdown protections:
++ Add option to check for a threshold number of servers to be registered and available on start-up 
+before the master begins tablet assignments when master acquires lock on start-up.  The options 
+allow for threshold number of servers and an optional max wait period.  
+See [Property Changes](#Property Change Summary) for property additions.
++ Add throttle for the number of shutdown requests sent by the master to tservers (Issue #1456)
+
+#### Optional GC metadata operations.
++ Added a property - gc.use.full.compaction At the completion of a GC cycle, the GC compacted the 
+metadata table to ensure that GC changes were flushed and persisted.  The property allows for the 
+action to be specified as compaction, flush or none.
+
+#### Update log statements:
+Compacting a table without user iterators was logging like it was an abnormal condition (#1347)
+Reduce verbose logging of merge operations in Master log (#1338)
+Improve logging for session expired events. When a tserver looses lock, the Session expired message
+is a log statement for every watcher - reduce current message to trace and add summary (#1108)
+
+#### Improve importdirectoy command
++ Add optional -t tablename to importdirectory shell command. (#1299)
+
+#### Add GC hadoop2 metrics reporting to expose GC  cycle metrics.
++ AccGcStarted - timestamp of GC cycle start 
++ AccGcFinished - timestamps of GC cycle finished 
++ AccGcCandidates - Number of candidates for GC 
++ AccGcInUse - Number of candidates still in use 
++ AccGcDeleted - Number of candidates deleted 
++ AccGcErrors - Number of  deletion errors 
++ AccGcWalStarted - timestamp of WAL collection start 
++ AccGcWalFinished - timestamp of WAL collection completion 
++ AccGcWalCandidates - number of WAL candidates for collection  
++ AccGcWalInUse  - number of WALs in use 
++ AccGcWalDeleted - number of WALs deleted 
++ AccGcWalErrors- number of errors during WAL deletion  
++ AccGcPostOpDuration - duration of compact / flush 
++ AccGcRunCycleCount - 1-up cycle count 
+
+#### Add option allow clone table to keep the clone offline.
++ Option to leave cloned tables offline (#1475) 
++ Updated the shell clone command with a -o option
+  
+#### Improved metadata table consistency checking and testing during GC
+    
++ During GC scans, an error will be thrown if the GC fails consistency checks (#1379)
++ Added a check to ensure the last tablet was seen.  This should cause an error to be thrown if 
+the scanner stops returning data for any reason.  
+
+### Java Lanaguage Level Changes
+This lists some of the code changes made to update the Java language level to Java 8.  They are
+listed here to aid migration of client code that may have similar issues migrating to the updated
+language level (#1467).
++ Update to maven compiler source and target to Java 1.8
++ Add optional dependency to pom for modernizer annotations
++ Add SuppressModernizer annotation to uses of Guava Predicate in AccumuloConfiguration and its 
+subclasses, Optional in VolumeChooser, and Function in GroupBalancer
++ Replace Guava Function, Predicate, Preconditions, Optional and Iterators where able with java.util equivalents
++ Replace core.util.Base64 class with java.util equivalent
++ Fix warnings introduced by Java 1.8
++ Drop method in MockConfiguration causing compiler warning and made PropertyFilter in 
+AccumuloConfiguration extend guava Predicate
++ Rename execute methods in MasterClient and ServerClient, like the 2.0 changes to fix the 
+overload warnings
++ Add colVis file to ContinuousInputFormat. Fixes #1368 (#1369)
+
+# Other Changes #
 
 Review comment:
   The other changes could just be a list like we did here:
   https://accumulo.apache.org/release/accumulo-1.9.3/#useful-links

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services