You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by ps...@apache.org on 2019/09/16 15:38:36 UTC
[hbase-operator-tools] tag 1.0.0RC0 created (now e40501f)
This is an automated email from the ASF dual-hosted git repository.
psomogyi pushed a change to tag 1.0.0RC0
in repository https://gitbox.apache.org/repos/asf/hbase-operator-tools.git.
at e40501f (commit)
This tag includes the following new commits:
new e40501f Preparing hbase-operator-tools release 1.0.0RC0; tagging and updates to CHANGES.md and RELEASENOTES.md
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[hbase-operator-tools] 01/01: Preparing hbase-operator-tools
release 1.0.0RC0; tagging and updates to CHANGES.md and RELEASENOTES.md
Posted by ps...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
psomogyi pushed a commit to tag 1.0.0RC0
in repository https://gitbox.apache.org/repos/asf/hbase-operator-tools.git
commit e40501fcf091c6ae28b6d5b3552df30b299b3ebc
Author: Peter Somogyi <ps...@apache.org>
AuthorDate: Mon Sep 16 15:38:27 2019 +0000
Preparing hbase-operator-tools release 1.0.0RC0; tagging and updates to CHANGES.md and RELEASENOTES.md
---
CHANGES.md | 17 +++-
RELEASENOTES.md | 151 ++++++++++++++++++++++++++++++++++
hbase-hbck2/pom.xml | 2 +-
hbase-operator-tools-assembly/pom.xml | 2 +-
pom.xml | 2 +-
5 files changed, 170 insertions(+), 4 deletions(-)
diff --git a/CHANGES.md b/CHANGES.md
index d12fbed..5f57a32 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -20,7 +20,7 @@
# Be careful doing manual edits in this file. Do not change format
# of release header or remove the below marker. This file is generated.
# DO NOT REMOVE THIS MARKER; FOR INTERPOLATING CHANGES!-->
-## Release hbase-operator-tools-1.0.0 - Unreleased (as of 2019-09-12)
+## Release hbase-operator-tools-1.0.0 - Unreleased (as of 2019-09-16)
@@ -28,6 +28,7 @@
| JIRA | Summary | Priority | Component |
|:---- |:---- | :--- |:---- |
+| [HBASE-22567](https://issues.apache.org/jira/browse/HBASE-22567) | [HBCK2] Add new methods for dealing with missing regions in META while Master is online | Major | hbck2 |
| [HBASE-22183](https://issues.apache.org/jira/browse/HBASE-22183) | [hbck2] Update hbck2 README to explain new "setRegionState" method. | Minor | documentation, hbck2 |
| [HBASE-22143](https://issues.apache.org/jira/browse/HBASE-22143) | HBCK2 setRegionState command | Minor | hbase-operator-tools, hbck2 |
@@ -36,6 +37,8 @@
| JIRA | Summary | Priority | Component |
|:---- |:---- | :--- |:---- |
+| [HBASE-22691](https://issues.apache.org/jira/browse/HBASE-22691) | [hbase-operator-tools] Move Checkstyle suppression file to different location | Trivial | hbck2 |
+| [HBASE-23018](https://issues.apache.org/jira/browse/HBASE-23018) | [HBCK2] Add useful messages when report/fixing missing regions in meta | Minor | hbase-operator-tools, hbck2, Operability |
| [HBASE-22999](https://issues.apache.org/jira/browse/HBASE-22999) | Fix non-varargs compile warnings | Minor | hbase-operator-tools |
@@ -43,6 +46,10 @@
| JIRA | Summary | Priority | Component |
|:---- |:---- | :--- |:---- |
+| [HBASE-23029](https://issues.apache.org/jira/browse/HBASE-23029) | Handle hbase-operator-tools releasenotes in release making script | Minor | hbase-operator-tools |
+| [HBASE-23026](https://issues.apache.org/jira/browse/HBASE-23026) | docker run command should not quote JAVA\_VOL | Major | hbase-operator-tools |
+| [HBASE-23025](https://issues.apache.org/jira/browse/HBASE-23025) | Do not quote GPG command | Major | hbase-operator-tools |
+| [HBASE-22984](https://issues.apache.org/jira/browse/HBASE-22984) | [HBCK2] HBCKMetaTableAccessor.deleteFromMetaTable throwing java.lang.UnsupportedOperationException at runtime | Major | hbck2 |
| [HBASE-22951](https://issues.apache.org/jira/browse/HBASE-22951) | [HBCK2] hbase hbck throws IOE "No FileSystem for scheme: hdfs" | Major | documentation, hbck2 |
| [HBASE-22952](https://issues.apache.org/jira/browse/HBASE-22952) | HBCK2 replication command is incompatible with 2.0.x | Critical | hbase-operator-tools |
| [HBASE-22949](https://issues.apache.org/jira/browse/HBASE-22949) | [HBCK2] Add lang3 as explicit dependency | Major | . |
@@ -52,6 +59,8 @@
| [HBASE-21484](https://issues.apache.org/jira/browse/HBASE-21484) | [HBCK2] hbck2 should default to a released hbase version | Major | hbck2 |
| [HBASE-21483](https://issues.apache.org/jira/browse/HBASE-21483) | [HBCK2] version string checking should look for exactly the version we know doesn't work | Major | hbck2 |
| [HBASE-21378](https://issues.apache.org/jira/browse/HBASE-21378) | [hbck2] add --skip version check to hbck2 tool (checkHBCKSupport blocks assigning hbase:meta or hbase:namespace when master is not initialized) | Major | hbase-operator-tools, hbck2 |
+| [HBASE-21335](https://issues.apache.org/jira/browse/HBASE-21335) | Change the default wait time of HBCK2 tool | Critical | . |
+| [HBASE-21317](https://issues.apache.org/jira/browse/HBASE-21317) | [hbck2] Add version, version handling, and misc override to assigns/unassigns | Major | hbase-operator-tools, hbck2 |
### TESTS:
@@ -65,8 +74,10 @@
| JIRA | Summary | Priority | Component |
|:---- |:---- | :--- |:---- |
+| [HBASE-23021](https://issues.apache.org/jira/browse/HBASE-23021) | [hbase-operator-tools] README edits in prep for release | Major | . |
| [HBASE-22997](https://issues.apache.org/jira/browse/HBASE-22997) | Move to SLF4J | Major | hbase-operator-tools |
| [HBASE-22998](https://issues.apache.org/jira/browse/HBASE-22998) | Fix NOTICE and LICENSE | Blocker | hbase-operator-tools |
+| [HBASE-22859](https://issues.apache.org/jira/browse/HBASE-22859) | [HBCK2] Fix the orphan regions on filesystem | Major | documentation, hbck2 |
| [HBASE-22825](https://issues.apache.org/jira/browse/HBASE-22825) | [HBCK2] Add a client-side to hbase-operator-tools that can exploit fixMeta added in server side | Major | hbck2 |
| [HBASE-22957](https://issues.apache.org/jira/browse/HBASE-22957) | [HBCK2] reference file check fails if compiled with old version but check against new | Major | . |
| [HBASE-22865](https://issues.apache.org/jira/browse/HBASE-22865) | [HBCK2] shows the whole help/usage message after the error message | Minor | hbck2 |
@@ -76,6 +87,8 @@
| [HBASE-21393](https://issues.apache.org/jira/browse/HBASE-21393) | Add an API ScheduleSCP() to HBCK2 | Major | hbase-operator-tools, hbck2 |
| [HBASE-22688](https://issues.apache.org/jira/browse/HBASE-22688) | [HBCK2] Add filesystem fixup to hbck2 | Major | hbck2 |
| [HBASE-22680](https://issues.apache.org/jira/browse/HBASE-22680) | [HBCK2] OfflineMetaRepair for hbase2/hbck2 | Major | hbck2 |
+| [HBASE-21322](https://issues.apache.org/jira/browse/HBASE-21322) | Add a scheduleServerCrashProcedure() API to HbckService | Critical | hbck2 |
+| [HBASE-21210](https://issues.apache.org/jira/browse/HBASE-21210) | Add bypassProcedure() API to HBCK2 | Major | hbase-operator-tools, hbck2 |
### OTHER:
@@ -83,6 +96,8 @@
| JIRA | Summary | Priority | Component |
|:---- |:---- | :--- |:---- |
| [HBASE-19121](https://issues.apache.org/jira/browse/HBASE-19121) | HBCK for AMv2 (A.K.A HBCK2) | Major | hbck, hbck2 |
+| [HBASE-23003](https://issues.apache.org/jira/browse/HBASE-23003) | [HBCK2/hbase-operator-tools] Release-making scripts | Major | . |
+| [HBASE-23002](https://issues.apache.org/jira/browse/HBASE-23002) | [HBCK2/hbase-operator-tools] Create an assembly that builds an hbase-operator-tools tgz | Major | . |
| [HBASE-22906](https://issues.apache.org/jira/browse/HBASE-22906) | Clean up checkstyle complaints in hbase-operator-tools | Trivial | hbase-operator-tools |
| [HBASE-22675](https://issues.apache.org/jira/browse/HBASE-22675) | Use commons-cli from hbase-thirdparty | Major | hbase-operator-tools |
| [HBASE-21433](https://issues.apache.org/jira/browse/HBASE-21433) | [hbase-operator-tools] Add Apache Yetus integration for hbase-operator-tools repository | Major | build, hbase-operator-tools |
diff --git a/RELEASENOTES.md b/RELEASENOTES.md
index d5ebc8f..bf0a547 100644
--- a/RELEASENOTES.md
+++ b/RELEASENOTES.md
@@ -27,6 +27,13 @@ These release notes cover new developer and user-facing incompatibilities, impor
---
+* [HBASE-23002](https://issues.apache.org/jira/browse/HBASE-23002) | *Major* | **[HBCK2/hbase-operator-tools] Create an assembly that builds an hbase-operator-tools tgz**
+
+First cut at an assembly for hbase-operator-tools project.
+
+
+---
+
* [HBASE-22997](https://issues.apache.org/jira/browse/HBASE-22997) | *Major* | **Move to SLF4J**
Added SLF4J binding for LOG4J 2.
@@ -34,6 +41,70 @@ Added SLF4J binding for LOG4J 2.
---
+* [HBASE-22567](https://issues.apache.org/jira/browse/HBASE-22567) | *Major* | **[HBCK2] Add new methods for dealing with missing regions in META while Master is online**
+
+Adds new, lightweight option for recovering missing regions in meta, "addFsRegionsMissingInMeta" command, as well "readonly" "reportMissingRegionsInMeta" to show the list of regions to be readded in meta. This is a less intrusive alternative to OfflineMetaRepair, can be used when master and meta are still capable of coming online.
+
+Detailed description as provided by the command usage help:
+
+{noformat}
+ addFsRegionsMissingInMeta \<NAMESPACE\|NAMESPACE:TABLENAME\>...
+ Options:
+ -d,--force\_disable aborts fix for table if disable fails.
+ To be used in scenarios where some regions may be missing in META,
+ but there's still a valid 'regioninfo' metadata file on HDFS.
+ This is a lighter version of 'OfflineMetaRepair tool commonly used for
+ similar issues on 1.x release line.
+ This command needs META to be online. For each table name passed as
+ parameter, it performs a diff between regions available in META,
+ against existing regions dirs on HDFS. Then, for region dirs with
+ no matches in META, it reads regioninfo metadata file and
+ re-creates given region in META. Regions are re-created in 'CLOSED'
+ state at META table only, but not in Masters' cache, and are not
+ assigned either. To get these regions online, run HBCK2
+'assigns'command
+ printed at the end of this command results for convenience.
+
+ NOTE: If using hbase releases older than 2.3.0, a rolling restart of
+ HMasters is needed prior to executing the provided 'assigns' command.
+
+ An example adding missing regions for tables 'tbl\_1' on default
+ namespace, 'tbl\_2' on namespace 'n1' and for all tables from
+ namespace 'n2':
+ $ HBCK2 addFsRegionsMissingInMeta default:tbl\_1 n1:tbl\_2 n2
+ Returns HBCK2 'assigns' command with all re-inserted regions.
+ SEE ALSO: reportMissingRegionsInMeta
+...
+ reportMissingRegionsInMeta \<NAMESPACE\|NAMESPACE:TABLENAME\>...
+ To be used in scenarios where some regions may be missing in META,
+ but there's still a valid 'regioninfo metadata file on HDFS.
+ This is a checking only method, designed for reporting purposes and
+ doesn't perform any fixes, providing a view of which regions (if any)
+ would get re-added to meta, grouped by respective table/namespace.
+ To effectively re-add regions in meta, addFsRegionsMissingInMeta should be executed.
+ This command needs META to be online. For each namespace/table passed
+ as parameter, it performs a diff between regions available in META,
+ against existing regions dirs on HDFS. Region dirs with no matches
+ are printed grouped under its related table name. Tables with no
+ missing regions will show a 'no missing regions' message. If no
+ namespace or table is specified, it will verify all existing regions.
+ It accepts a combination of multiple namespace and tables. Table names
+ should include the namespace portion, even for tables in the default
+ namespace, otherwise it will assume as a namespace value.
+ An example triggering missing regions report for tables 'table\_1'
+ and 'table\_2', under default namespace:
+ $ HBCK2 reportMissingRegionsInMeta default:table\_1 default:table\_2
+ An example triggering missing regions report for table 'table\_1'
+ under default namespace, and for all tables from namespace 'ns1':
+ $ HBCK2 reportMissingRegionsInMeta default:table\_1 ns1
+ Returns list of missing regions for each table passed as parameter, or
+ for each table on namespaces specified as parameter.
+...
+{noformat}
+
+
+---
+
* [HBASE-22717](https://issues.apache.org/jira/browse/HBASE-22717) | *Major* | **[HBCK2] Expose replication fixes from hbck1**
Adds 'replication' command to HBCK2. Will clear old deleted peer queues and if a table name is passed, can clear replication barrier flags.
@@ -79,6 +150,13 @@ Given the encoded name of a Region, this command allows you to change the state
---
+* [HBASE-21322](https://issues.apache.org/jira/browse/HBASE-21322) | *Critical* | **Add a scheduleServerCrashProcedure() API to HbckService**
+
+Adds scheduleServerCrashProcedure to the HbckService.
+
+
+---
+
* [HBASE-21378](https://issues.apache.org/jira/browse/HBASE-21378) | *Major* | **[hbck2] add --skip version check to hbck2 tool (checkHBCKSupport blocks assigning hbase:meta or hbase:namespace when master is not initialized)**
Adds a general -s,--skip option to hbck2 so you can run the command it first checking it is version compatible.
@@ -86,4 +164,77 @@ Adds a general -s,--skip option to hbck2 so you can run the command it first che
Should not be needed going forward but in the spirt of our not knowing all the conditions under which we'll be trying to run hbck2, adding it.
+---
+
+* [HBASE-21335](https://issues.apache.org/jira/browse/HBASE-21335) | *Critical* | **Change the default wait time of HBCK2 tool**
+
+Changed waitTime parameter to lockWait on bypass. Changed default waitTime from 0 -- i.e. wait for ever -- to 1ms so if lock is held, we'll go past it and if override enforce bypass.
+
+
+---
+
+* [HBASE-21317](https://issues.apache.org/jira/browse/HBASE-21317) | *Major* | **[hbck2] Add version, version handling, and misc override to assigns/unassigns**
+
+Adds --override to assigns and unassigns.
+Adds --recursive to bypass
+Adds dump of version info
+Adds fail if remote cluster doesn't support hbck2
+
+
+---
+
+* [HBASE-21210](https://issues.apache.org/jira/browse/HBASE-21210) | *Major* | **Add bypassProcedure() API to HBCK2**
+
+Added a bypass to hbck2:
+
+{code}
+$ HBASE\_CLASSPATH\_PREFIX=../hbase-operator-tools/hbase-hbck2/target/hbase-hbck2-1.0.0-SNAPSHOT.jar ./bin/hbase org.apache.hbase.HBCK2
+usage: HBCK2 [OPTIONS] COMMAND \<ARGS\>
+
+Options:
+ -d,--debug run with debug output
+ -h,--help output this help message
+ -p,--hbase.zookeeper.property.clientPort peerport of target hbase
+ ensemble
+ -q,--hbase.zookeeper.quorum \<arg\> ensemble of target hbase
+ -z,--zookeeper.znode.parent parent znode of target hbase
+
+Commands:
+ setTableState \<TABLENAME\> \<STATE\>
+ Possible table states: ENABLED, DISABLED, DISABLING, ENABLING
+ To read current table state, in the hbase shell run:
+ hbase\> get 'hbase:meta', '\<TABLENAME\>', 'table:state'
+ A value of \\x08\\x00 == ENABLED, \\x08\\x01 == DISABLED, etc.
+ An example making table name 'user' ENABLED:
+ $ HBCK2 setTableState users ENABLED
+ Returns whatever the previous table state was.
+
+ assigns \<ENCODED\_REGIONNAME\>...
+ A 'raw' assign that can be used even during Master initialization.
+ Skirts Coprocessors. Pass one or more encoded RegionNames:
+ e.g. 1588230740 is hard-coded encoding for hbase:meta region and
+ de00010733901a05f5a2a3a382e27dd4 is an example of what a random
+ user-space encoded Region name looks like. For example:
+ $ HBCK2 assign 1588230740 de00010733901a05f5a2a3a382e27dd4
+ Returns the pid of the created AssignProcedure or -1 if none.
+
+ bypass [OPTIONS] \<PID\>...
+ Pass one (or more) procedure 'pid's to skip to the procedure finish.
+ Parent of this procedures will also skip to its finish. Entities will
+ be left in an inconsistent state and will require manual fixup.
+ Pass --force to break any outstanding locks.
+ Pass --waitTime=\<seconds\> to wait on entity lock before giving up.
+ Default: force=false and waitTime=0. Returns true if succeeded.
+
+ unassigns \<ENCODED\_REGIONNAME\>...
+ A 'raw' unassign that can be used even during Master initialization.
+ Skirts Coprocessors. Pass one or more encoded RegionNames:
+ Skirts Coprocessors. Pass one or more encoded RegionNames:
+ de00010733901a05f5a2a3a382e27dd4 is an example of what a random
+ user-space encoded Region name looks like. For example:
+ $ HBCK2 unassign 1588230740 de00010733901a05f5a2a3a382e27dd4
+ Returns the pid of the created UnassignProcedure or -1 if none.
+{code}
+
+
diff --git a/hbase-hbck2/pom.xml b/hbase-hbck2/pom.xml
index 13ff9d0..1be4111 100644
--- a/hbase-hbck2/pom.xml
+++ b/hbase-hbck2/pom.xml
@@ -24,7 +24,7 @@
<parent>
<groupId>org.apache.hbase.operator.tools</groupId>
<artifactId>hbase-operator-tools</artifactId>
- <version>1.0.0-SNAPSHOT</version>
+ <version>1.0.0</version>
<relativePath>..</relativePath>
</parent>
<artifactId>hbase-hbck2</artifactId>
diff --git a/hbase-operator-tools-assembly/pom.xml b/hbase-operator-tools-assembly/pom.xml
index d282929..cf37b0a 100644
--- a/hbase-operator-tools-assembly/pom.xml
+++ b/hbase-operator-tools-assembly/pom.xml
@@ -23,7 +23,7 @@
<parent>
<groupId>org.apache.hbase.operator.tools</groupId>
<artifactId>hbase-operator-tools</artifactId>
- <version>1.0.0-SNAPSHOT</version>
+ <version>1.0.0</version>
<relativePath>..</relativePath>
</parent>
<artifactId>hbase-operator-tools-assembly</artifactId>
diff --git a/pom.xml b/pom.xml
index b228376..6d08d70 100644
--- a/pom.xml
+++ b/pom.xml
@@ -38,7 +38,7 @@
</parent>
<groupId>org.apache.hbase.operator.tools</groupId>
<artifactId>hbase-operator-tools</artifactId>
- <version>1.0.0-SNAPSHOT</version>
+ <version>1.0.0</version>
<name>Apache HBase Operator Tools</name>
<packaging>pom</packaging>
<description>