You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by zg...@apache.org on 2020/05/21 10:03:55 UTC
[hbase] branch branch-2.2 updated: HBASE-24410 Generate CHANGES.md
and RELEASENOTES.md for 2.2.5 (#1752)
This is an automated email from the ASF dual-hosted git repository.
zghao pushed a commit to branch branch-2.2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.2 by this push:
new 9b04812 HBASE-24410 Generate CHANGES.md and RELEASENOTES.md for 2.2.5 (#1752)
9b04812 is described below
commit 9b048121a307306cf14f0b1ff27493e38834bca0
Author: Guanghao Zhang <zg...@apache.org>
AuthorDate: Thu May 21 18:03:34 2020 +0800
HBASE-24410 Generate CHANGES.md and RELEASENOTES.md for 2.2.5 (#1752)
---
CHANGES.md | 118 +++++++++++++++++++++++++++++++++++++++++++++++++++++
RELEASENOTES.md | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 242 insertions(+)
diff --git a/CHANGES.md b/CHANGES.md
index c44299b..48caf3b 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -20,6 +20,124 @@
# Be careful doing manual edits in this file. Do not change format
# of release header or remove the below marker. This file is generated.
# DO NOT REMOVE THIS MARKER; FOR INTERPOLATING CHANGES!-->
+## Release 2.2.5 - Unreleased (as of 2020-05-21)
+
+### IMPROVEMENTS:
+
+| JIRA | Summary | Priority | Component |
+|:---- |:---- | :--- |:---- |
+| [HBASE-24386](https://issues.apache.org/jira/browse/HBASE-24386) | TableSnapshotScanner support scan limit | Major | Scanners, snapshots |
+| [HBASE-24164](https://issues.apache.org/jira/browse/HBASE-24164) | Retain the ReadRequests and WriteRequests of region on web UI after alter table | Major | metrics |
+| [HBASE-24355](https://issues.apache.org/jira/browse/HBASE-24355) | Fix typos in the HStore#compact annotation | Minor | . |
+| [HBASE-24345](https://issues.apache.org/jira/browse/HBASE-24345) | [ACL] renameRSGroup should require Admin level permission | Major | acl, rsgroup |
+| [HBASE-24328](https://issues.apache.org/jira/browse/HBASE-24328) | skip duplicate GCMultipleMergedRegionsProcedure while previous finished | Major | . |
+| [HBASE-24221](https://issues.apache.org/jira/browse/HBASE-24221) | Support bulkLoadHFile by family | Major | HFile |
+| [HBASE-24302](https://issues.apache.org/jira/browse/HBASE-24302) | Add an "ignoreTimestamps" option (defaulted to false) to HashTable/SyncTable tool | Major | . |
+| [HBASE-24199](https://issues.apache.org/jira/browse/HBASE-24199) | Procedure related metrics is not consumed in the JMX metric | Minor | metrics |
+| [HBASE-24252](https://issues.apache.org/jira/browse/HBASE-24252) | Implement proxyuser/doAs mechanism for hbase-http | Major | security, UI |
+| [HBASE-23264](https://issues.apache.org/jira/browse/HBASE-23264) | Resolve a TODO of BucketAllocator: "Why we add the extra 1024 bytes? Slop?" | Minor | BucketCache |
+| [HBASE-24222](https://issues.apache.org/jira/browse/HBASE-24222) | remove FSUtils.checkAccess and replace with FileSystem.access in HBCK | Major | Filesystem Integration |
+| [HBASE-24196](https://issues.apache.org/jira/browse/HBASE-24196) | [Shell] Add rename rsgroup command in hbase shell | Major | rsgroup, shell |
+| [HBASE-24166](https://issues.apache.org/jira/browse/HBASE-24166) | Duplicate implementation for acquireLock between CreateTableProcedure and its parent class | Minor | proc-v2 |
+| [HBASE-24148](https://issues.apache.org/jira/browse/HBASE-24148) | Upgrade Thrift to 0.13.0: 0.12.0 has outstanding CVEs. | Major | Thrift |
+| [HBASE-24112](https://issues.apache.org/jira/browse/HBASE-24112) | [RSGroup] Support renaming rsgroup | Major | rsgroup |
+| [HBASE-24181](https://issues.apache.org/jira/browse/HBASE-24181) | Add region info when log meessages in HRegion. | Minor | regionserver |
+| [HBASE-24099](https://issues.apache.org/jira/browse/HBASE-24099) | Use a fair ReentrantReadWriteLock for the region close lock | Major | . |
+| [HBASE-24077](https://issues.apache.org/jira/browse/HBASE-24077) | When encounter RowTooBigException, log the row info. | Minor | . |
+| [HBASE-24111](https://issues.apache.org/jira/browse/HBASE-24111) | Enable CompactionTool executions on non-HDFS filesystems | Major | Compaction, mapreduce, tooling |
+| [HBASE-24102](https://issues.apache.org/jira/browse/HBASE-24102) | RegionMover should exclude draining/decommissioning nodes from target RSs | Major | . |
+| [HBASE-24021](https://issues.apache.org/jira/browse/HBASE-24021) | Fail fast when bulkLoadHFiles method catch some IOException | Major | HFile, regionserver |
+| [HBASE-8868](https://issues.apache.org/jira/browse/HBASE-8868) | add metric to report client shortcircuit reads | Minor | metrics, regionserver |
+| [HBASE-24032](https://issues.apache.org/jira/browse/HBASE-24032) | [RSGroup] Assign created tables to respective rsgroup automatically instead of manual operations | Major | master, rsgroup |
+| [HBASE-23967](https://issues.apache.org/jira/browse/HBASE-23967) | Improve the accuracy of the method sizeToString | Minor | . |
+
+
+### BUG FIXES:
+
+| JIRA | Summary | Priority | Component |
+|:---- |:---- | :--- |:---- |
+| [HBASE-24364](https://issues.apache.org/jira/browse/HBASE-24364) | [Chaos Monkey] Invalid data block encoding in ChangeEncodingAction | Major | . |
+| [HBASE-24399](https://issues.apache.org/jira/browse/HBASE-24399) | [Flakey Tests] Some UTs about RSGroup should wait RSGroupInfoManager to be online | Minor | rsgroup |
+| [HBASE-20289](https://issues.apache.org/jira/browse/HBASE-20289) | Comparator for NormalizationPlan breaks comparator's convention | Minor | master |
+| [HBASE-24365](https://issues.apache.org/jira/browse/HBASE-24365) | MetricsTableWrapperAggregateImpl runnable fails due to exception and never runs | Minor | metrics |
+| [HBASE-24377](https://issues.apache.org/jira/browse/HBASE-24377) | MemStoreFlusher throw NullPointerException | Major | . |
+| [HBASE-24381](https://issues.apache.org/jira/browse/HBASE-24381) | The Size metrics in Master Webui is wrong if the size is 0 | Major | UI |
+| [HBASE-24135](https://issues.apache.org/jira/browse/HBASE-24135) | TableStateNotFoundException happends when table creation if rsgroup is enable | Minor | . |
+| [HBASE-24165](https://issues.apache.org/jira/browse/HBASE-24165) | maxPoolSize is logged incorrectly in ByteBufferPool | Minor | . |
+| [HBASE-23702](https://issues.apache.org/jira/browse/HBASE-23702) | \`hbase.hstore.flusher.count\` setting to 0 breaks HMaster | Major | regionserver |
+| [HBASE-23832](https://issues.apache.org/jira/browse/HBASE-23832) | Old config hbase.hstore.compactionThreshold is ignored | Critical | . |
+| [HBASE-24190](https://issues.apache.org/jira/browse/HBASE-24190) | Make kerberos value of hbase.security.authentication property case insensitive | Major | . |
+| [HBASE-24363](https://issues.apache.org/jira/browse/HBASE-24363) | Fix failed ut TestAssignmentManagerMetrics for branch-2.2 | Major | . |
+| [HBASE-24349](https://issues.apache.org/jira/browse/HBASE-24349) | Use rubocop 0.81.0 in Dockerfile | Blocker | build |
+| [HBASE-24273](https://issues.apache.org/jira/browse/HBASE-24273) | HBCK's "Orphan Regions on FileSystem" reports regions with referenced HFiles | Critical | hbck2 |
+| [HBASE-24322](https://issues.apache.org/jira/browse/HBASE-24322) | UnsafeAvailChecker should also check that required methods are available | Minor | . |
+| [HBASE-24314](https://issues.apache.org/jira/browse/HBASE-24314) | Some classes still use log4j logger directly | Blocker | logging |
+| [HBASE-24288](https://issues.apache.org/jira/browse/HBASE-24288) | Allow admin user to create table and do bulkLoad | Major | acl |
+| [HBASE-22710](https://issues.apache.org/jira/browse/HBASE-22710) | Wrong result in one case of scan that use raw and versions and filter together | Major | Scanners |
+| [HBASE-24282](https://issues.apache.org/jira/browse/HBASE-24282) | 'scandetail' log message is missing when responseTooSlow happens on the first scan rpc call | Major | Operability |
+| [HBASE-24274](https://issues.apache.org/jira/browse/HBASE-24274) | \`RESTApiClusterManager\` attempts to deserialize response using serialization API | Major | integration tests |
+| [HBASE-24281](https://issues.apache.org/jira/browse/HBASE-24281) | Remove some stale hadoop.version properties in the pom for sub modules | Major | hadoop3, pom |
+| [HBASE-24213](https://issues.apache.org/jira/browse/HBASE-24213) | Backport the pre commit changes for HBASE-24169 to all branches | Major | scripts |
+| [HBASE-24174](https://issues.apache.org/jira/browse/HBASE-24174) | Fix findbugs warning on ServiceAuthorizationManager | Major | . |
+| [HBASE-24177](https://issues.apache.org/jira/browse/HBASE-24177) | MetricsTable#updateFlushTime is wrong | Minor | metrics |
+| [HBASE-23833](https://issues.apache.org/jira/browse/HBASE-23833) | The relocated hadoop-thirdparty protobuf breaks HBase asyncwal | Major | wal |
+| [HBASE-24158](https://issues.apache.org/jira/browse/HBASE-24158) | [Flakey Tests] TestAsyncTableGetMultiThreaded | Major | . |
+| [HBASE-24186](https://issues.apache.org/jira/browse/HBASE-24186) | RegionMover ignores replicationId | Minor | read replicas |
+| [HBASE-24151](https://issues.apache.org/jira/browse/HBASE-24151) | [rsgroup] The master server aborted for IllegalThreadStateException | Minor | rsgroup |
+| [HBASE-24197](https://issues.apache.org/jira/browse/HBASE-24197) | TestHttpServer.testBindAddress failure with latest jetty | Major | . |
+| [HBASE-24176](https://issues.apache.org/jira/browse/HBASE-24176) | user\_permission '.\*' command failed to show all table permissions | Minor | shell |
+| [HBASE-24167](https://issues.apache.org/jira/browse/HBASE-24167) | [Flakey Tests] TestHRegionWithInMemoryFlush#testWritesWhileScanning gets stuck MVCC | Major | . |
+| [HBASE-23998](https://issues.apache.org/jira/browse/HBASE-23998) | Update license for jetty-client | Major | build, dependencies |
+| [HBASE-24074](https://issues.apache.org/jira/browse/HBASE-24074) | ConcurrentModificationException occurred in ReplicationSourceManager while refreshing the peer | Major | Replication |
+| [HBASE-24121](https://issues.apache.org/jira/browse/HBASE-24121) | [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be. | Major | rpc, security |
+| [HBASE-24122](https://issues.apache.org/jira/browse/HBASE-24122) | Change machine ulimit-l to ulimit-a so dumps full ulimit rather than just 'max locked memory' | Major | build |
+| [HBASE-24041](https://issues.apache.org/jira/browse/HBASE-24041) | [regression] Increase RESTServer buffer size back to 64k | Major | REST |
+| [HBASE-24057](https://issues.apache.org/jira/browse/HBASE-24057) | Add modules to mapreduce job classpaths | Major | mapreduce |
+| [HBASE-24040](https://issues.apache.org/jira/browse/HBASE-24040) | WALFactory.Providers.multiwal causes StackOverflowError | Minor | wal |
+| [HBASE-23949](https://issues.apache.org/jira/browse/HBASE-23949) | refactor loadBalancer implements for rsgroup balance by table to achieve overallbalanced | Major | rsgroup |
+| [HBASE-24043](https://issues.apache.org/jira/browse/HBASE-24043) | [Flakey Tests] TestAsyncRegionAdminApi, TestRegionMergeTransactionOnCluster fixes and debug | Major | flakies |
+| [HBASE-24019](https://issues.apache.org/jira/browse/HBASE-24019) | Correct exception messages for table null and namespace unavailable. | Minor | . |
+| [HBASE-24016](https://issues.apache.org/jira/browse/HBASE-24016) | Change nightly poll from cron @daily to pollSCM @daily; i.e. run nightly if a change ONLY | Major | . |
+| [HBASE-22103](https://issues.apache.org/jira/browse/HBASE-22103) | HDFS-13209 in Hadoop 3.3.0 breaks asyncwal | Major | hadoop3, wal |
+
+
+### TESTS:
+
+| JIRA | Summary | Priority | Component |
+|:---- |:---- | :--- |:---- |
+| [HBASE-23957](https://issues.apache.org/jira/browse/HBASE-23957) | [flakey test] client.TestMultiParallel fails to read hbase-site.xml | Minor | test |
+| [HBASE-24080](https://issues.apache.org/jira/browse/HBASE-24080) | [flakey test] TestRegionReplicaFailover.testSecondaryRegionKill fails. | Major | read replicas |
+| [HBASE-23943](https://issues.apache.org/jira/browse/HBASE-23943) | Rubocop configuration needs updated | Minor | build |
+| [HBASE-24115](https://issues.apache.org/jira/browse/HBASE-24115) | Relocate test-only REST "client" from src/ to test/ and mark Private | Major | REST, security |
+| [HBASE-23999](https://issues.apache.org/jira/browse/HBASE-23999) | [flakey test] TestTableOutputFormatConnectionExhaust | Major | test |
+
+
+### SUB-TASKS:
+
+| JIRA | Summary | Priority | Component |
+|:---- |:---- | :--- |:---- |
+| [HBASE-24218](https://issues.apache.org/jira/browse/HBASE-24218) | Add hadoop 3.2.x in hadoop check | Major | scripts |
+| [HBASE-24175](https://issues.apache.org/jira/browse/HBASE-24175) | [Flakey Tests] TestSecureExportSnapshot FileNotFoundException | Major | flakies |
+| [HBASE-23992](https://issues.apache.org/jira/browse/HBASE-23992) | Fix TestAdminShell and TestQuotasShell mistakenly broken by parent commit | Trivial | test |
+| [HBASE-24050](https://issues.apache.org/jira/browse/HBASE-24050) | Deprecated PBType on all 2.x branches | Major | Protobufs |
+| [HBASE-24022](https://issues.apache.org/jira/browse/HBASE-24022) | Set version as 2.2.5-SNAPSHOT in branch-2.2 | Major | . |
+| [HBASE-24009](https://issues.apache.org/jira/browse/HBASE-24009) | Backport the personality changes in HBASE-23989 to all active branches | Major | build |
+| [HBASE-23891](https://issues.apache.org/jira/browse/HBASE-23891) | Add an option to Actions to filter out meta RS | Minor | integration tests |
+
+
+### OTHER:
+
+| JIRA | Summary | Priority | Component |
+|:---- |:---- | :--- |:---- |
+| [HBASE-24271](https://issues.apache.org/jira/browse/HBASE-24271) | Set values in \`conf/hbase-site.xml\` that enable running on \`LocalFileSystem\` out of the box | Major | . |
+| [HBASE-24258](https://issues.apache.org/jira/browse/HBASE-24258) | [Hadoop3.3] Update license for org.ow2.asm:\* | Minor | dependencies |
+| [HBASE-24301](https://issues.apache.org/jira/browse/HBASE-24301) | Update Apache POM to version 23 | Minor | . |
+| [HBASE-23896](https://issues.apache.org/jira/browse/HBASE-23896) | Snapshot owner cannot delete snapshot when ACL is enabled and Kerberos is not enabled | Major | . |
+| [HBASE-24138](https://issues.apache.org/jira/browse/HBASE-24138) | Ensure StochasticLoadBalancer can log details of decision to not run balancer | Major | Balancer, Operability |
+| [HBASE-23047](https://issues.apache.org/jira/browse/HBASE-23047) | ChecksumUtil.validateChecksum logs an INFO message inside a "if(LOG.isTraceEnabled())" block. | Minor | . |
+
+
+
## Release 2.2.4 - Unreleased (as of 2020-03-11)
### NEW FEATURES:
diff --git a/RELEASENOTES.md b/RELEASENOTES.md
index 369b514..e449d1c 100644
--- a/RELEASENOTES.md
+++ b/RELEASENOTES.md
@@ -20,6 +20,130 @@
# Be careful doing manual edits in this file. Do not change format
# of release header or remove the below marker. This file is generated.
# DO NOT REMOVE THIS MARKER; FOR INTERPOLATING CHANGES!-->
+# HBASE 2.2.5 Release Notes
+
+These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements.
+
+---
+
+* [HBASE-24271](https://issues.apache.org/jira/browse/HBASE-24271) | *Major* | **Set values in \`conf/hbase-site.xml\` that enable running on \`LocalFileSystem\` out of the box**
+
+<!-- markdown -->
+HBASE-24271 makes changes the the default `conf/hbase-site.xml` such that `bin/hbase` will run directly out of the binary tarball or a compiled source tree without any configuration modifications vs. Hadoop 2.8+. This changes our long-standing history of shipping no configured values in `conf/hbase-site.xml`, so existing processes that assume this file is empty of configuration properties may require attention.
+
+
+---
+
+* [HBASE-22710](https://issues.apache.org/jira/browse/HBASE-22710) | *Major* | **Wrong result in one case of scan that use raw and versions and filter together**
+
+Make the logic of the versions chosen more reasonable for raw scan, to avoid lose result when using filter.
+
+
+---
+
+* [HBASE-24252](https://issues.apache.org/jira/browse/HBASE-24252) | *Major* | **Implement proxyuser/doAs mechanism for hbase-http**
+
+This feature enables the HBase Web UI's to accept a 'proxyuser' via the HTTP Request's query string. When the parameter \`hbase.security.authentication.spnego.kerberos.proxyuser.enable\` is set to \`true\` in hbase-site.xml (default is \`false\`), the HBase UI will attempt to impersonate the user specified by the query parameter "doAs". This query parameter is checked case-insensitively. When this option is not provided, the user who executed the request is the "real" user and there is n [...]
+
+For example, if the user "bob" with Kerberos credentials executes a request against the WebUI with this feature enabled and a query string which includes \`doAs=alice\`, the HBase UI will treat this request as executed as \`alice\`, not \`bob\`.
+
+The standard Hadoop proxyuser configuration properties to limit users who may impersonate others apply to this change (e.g. to enable \`bob\` to impersonate \`alice\`). See the Hadoop documentation for more information on how to configure these proxyuser rules.
+
+
+---
+
+* [HBASE-24196](https://issues.apache.org/jira/browse/HBASE-24196) | *Major* | **[Shell] Add rename rsgroup command in hbase shell**
+
+user or admin can now use
+hbase shell \> rename\_rsgroup 'oldname', 'newname'
+to rename rsgroup.
+
+
+---
+
+* [HBASE-24218](https://issues.apache.org/jira/browse/HBASE-24218) | *Major* | **Add hadoop 3.2.x in hadoop check**
+
+Add hadoop-3.2.0 and hadoop-3.2.1 in hadoop check and when '--quick-hadoopcheck' we will only check hadoop-3.2.1.
+
+Notice that, for aligning the personality scripts across all the active branches, we will commit the patch to all active branches, but the hadoop-3.2.x support in hadoopcheck is only applied to branch-2.2+.
+
+
+---
+
+* [HBASE-24112](https://issues.apache.org/jira/browse/HBASE-24112) | *Major* | **[RSGroup] Support renaming rsgroup**
+
+Support RSGroup renaming in core codebase. New API Admin#renameRSGroup(String, String) is introduced in 3.0.0.
+
+
+---
+
+* [HBASE-24121](https://issues.apache.org/jira/browse/HBASE-24121) | *Major* | **[Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be.**
+
+Master & RegionService now support refresh policy authorization defined in hbase-policy.xml without restarting service. To refresh policy, please execute hbase shell command: update\_config or update\_config\_all after policy file updated and synced on all nodes.
+
+
+---
+
+* [HBASE-24099](https://issues.apache.org/jira/browse/HBASE-24099) | *Major* | **Use a fair ReentrantReadWriteLock for the region close lock**
+
+This change modifies the default acquisition policy for the region's close lock in order to prevent observed starvation of close requests. The new boolean configuration parameter 'hbase.regionserver.fair.region.close.lock' controls the lock acquisition policy: if true, the lock is created in fair mode (default); if false, the lock is created in nonfair mode (the old default).
+
+
+---
+
+* [HBASE-24122](https://issues.apache.org/jira/browse/HBASE-24122) | *Major* | **Change machine ulimit-l to ulimit-a so dumps full ulimit rather than just 'max locked memory'**
+
+Our 'Build Artifacts' have a machine directory under which we emit vitals on the host the build was run on. We used to emit the result of 'ulimit -l' as a file named 'ulimit-l'. This has been hijacked to instead emit result of running 'ulimit -a' which includes stat on ulimit -l.
+
+
+---
+
+* [HBASE-24050](https://issues.apache.org/jira/browse/HBASE-24050) | *Major* | **Deprecated PBType on all 2.x branches**
+
+org.apache.hadoop.hbase.types.PBType is marked as deprecated without any replacement. It will be moved to hbase-example module and marked as IA.Private in 3.0.0. This is a mistake as it should not be part of our public API. Users who depend on this class should just copy the code your own code base.
+
+
+---
+
+* [HBASE-8868](https://issues.apache.org/jira/browse/HBASE-8868) | *Minor* | **add metric to report client shortcircuit reads**
+
+Expose file system level read metrics for RegionServer.
+
+If the HBase RS runs on top of HDFS, calculate the aggregation of
+ReadStatistics of each HdfsFileInputStream. These metrics include:
+(1) total number of bytes read from HDFS.
+(2) total number of bytes read from local DataNode.
+(3) total number of bytes read locally through short-circuit read.
+(4) total number of bytes read locally through zero-copy read.
+
+Because HDFS ReadStatistics is calculated per input stream, it is not
+feasible to update the aggregated number in real time. Instead, the
+metrics are updated when an input stream is closed.
+
+
+---
+
+* [HBASE-24032](https://issues.apache.org/jira/browse/HBASE-24032) | *Major* | **[RSGroup] Assign created tables to respective rsgroup automatically instead of manual operations**
+
+Admin can determine which tables go to which rsgroup by script (setting hbase.rsgroup.table.mapping.script with local filystem path) on Master side which aims to lighten the burden of admin operations. Note, since HBase 3+, rsgroup can be specified in TableDescriptor as well, if clients specify this, master will skip the determination from script.
+
+Here is a simple example of script:
+{code}
+# Input consists of two string, 1st is the namespace of the table, 2nd is the table name of the table
+#!/bin/bash
+namespace=$1
+tablename=$2
+if [[ $namespace == test ]]; then
+ echo test
+elif [[ $tablename == \*foo\* ]]; then
+ echo other
+else
+ echo default
+fi
+{code}
+
+
+
# HBASE 2.2.4 Release Notes
These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements.