You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by ap...@apache.org on 2019/06/05 22:55:35 UTC
[hbase] branch branch-1.3 updated (da2d2a9 -> b59afe7)
This is an automated email from the ASF dual-hosted git repository.
apurtell pushed a change to branch branch-1.3
in repository https://gitbox.apache.org/repos/asf/hbase.git.
discard da2d2a9 Update CHANGES.txt for 1.3.5RC0
discard cade234 HBASE-22540 [Memstore] Correct counters in MemStoreChunkPool
new 9ae9b5b Amend HBASE-21920 Ignoring 'empty' end_key while calculating end_key for new region in HBCK -fixHdfsOverlaps command can cause data loss
new 9973601 HBASE-22540 [Memstore] Correct counters in MemStoreChunkPool
new b59afe7 Update CHANGES.txt for 1.3.5RC0
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (da2d2a9)
\
N -- N -- N refs/heads/branch-1.3 (b59afe7)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../java/org/apache/hadoop/hbase/HBaseTestingUtility.java | 15 +++++++++++++++
1 file changed, 15 insertions(+)
[hbase] 01/03: Amend HBASE-21920 Ignoring 'empty' end_key while
calculating end_key for new region in HBCK -fixHdfsOverlaps command can
cause data loss
Posted by ap...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
apurtell pushed a commit to branch branch-1.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 9ae9b5b03ba236b72c047f0f0d92cd50e57d8a0d
Author: Andrew Purtell <ap...@apache.org>
AuthorDate: Wed Jun 5 15:48:33 2019 -0700
Amend HBASE-21920 Ignoring 'empty' end_key while calculating end_key for new region in HBCK -fixHdfsOverlaps command can cause data loss
Add missing test support method HTU#assignRegion
---
.../java/org/apache/hadoop/hbase/HBaseTestingUtility.java | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java b/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
index 6a33a19..cbd8296 100644
--- a/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
+++ b/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
@@ -3454,6 +3454,21 @@ public class HBaseTestingUtility extends HBaseCommonTestingUtility {
}
/**
+ * Uses directly the assignment manager to assign the region.
+ * and waits until the specified region has completed assignment.
+ * @param tableName the table name
+ * @throws IOException
+ * @throw InterruptedException
+ * @return true if the region is assigned false otherwise.
+ */
+ public boolean assignRegion(final HRegionInfo regionInfo)
+ throws IOException, InterruptedException {
+ final AssignmentManager am = getHBaseCluster().getMaster().getAssignmentManager();
+ am.assign(regionInfo, true);
+ return am.waitForAssignment(regionInfo);
+ }
+
+ /**
* Wait until all regions for a table in hbase:meta have a non-empty
* info:server, up to a configuable timeout value (default is 60 seconds)
* This means all regions have been deployed,
[hbase] 03/03: Update CHANGES.txt for 1.3.5RC0
Posted by ap...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
apurtell pushed a commit to branch branch-1.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit b59afe7b1dc650ff3a86034477b563734e8799a9
Author: Andrew Purtell <ap...@apache.org>
AuthorDate: Wed Jun 5 15:35:10 2019 -0700
Update CHANGES.txt for 1.3.5RC0
---
CHANGES.txt | 2 ++
1 file changed, 2 insertions(+)
diff --git a/CHANGES.txt b/CHANGES.txt
index 6c5bb84..13e57db 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -11,6 +11,7 @@ Release Notes - HBase - Version 1.3.5 06/14/2019
** Bug
* [HBASE-17170] - HBase is also retrying DoNotRetryIOException because of class loader differences.
* [HBASE-20851] - Change rubocop config for max line length of 100
+ * [HBASE-21920] - Ignoring 'empty' end_key while calculating end_key for new region in HBCK -fixHdfsOverlaps command can cause data loss
* [HBASE-22235] - OperationStatus.{SUCCESS|FAILURE|NOT_RUN} are not visible to 3rd party coprocessors
* [HBASE-22270] - master's jmx.clusterRequests could be negative in branch-1
* [HBASE-22391] - Fix flaky tests from TestFromClientSide
@@ -20,6 +21,7 @@ Release Notes - HBase - Version 1.3.5 06/14/2019
* [HBASE-22508] - Address findbugs/spotbugs complaints (branch-1.3)
* [HBASE-22518] - yetus personality is treating branch-1.4 like earlier branches for hadoopcheck
* [HBASE-22533] - TestAccessController3 is flaky (branch-1)
+ * [HBASE-22540] - [Memstore] Correct counters in MemStoreChunkPool
** New Feature
* [HBASE-21815] - Make isTrackingMetrics and getMetrics of ScannerContext public
[hbase] 02/03: HBASE-22540 [Memstore] Correct counters in
MemStoreChunkPool
Posted by ap...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
apurtell pushed a commit to branch branch-1.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 9973601b284d4a97d33cbdb95d32ca7303bbf6c4
Author: Reid Chan <re...@apache.org>
AuthorDate: Wed Jun 5 11:55:10 2019 +0800
HBASE-22540 [Memstore] Correct counters in MemStoreChunkPool
* First commit: HBASE-22540 [Memstore] Correct counters in MemStoreChunkPool
* Address comment and remove useless parent thread info from Statistics thread
Signed-off-by: Andrew Purtell <ap...@apache.org>
---
.../hbase/regionserver/MemStoreChunkPool.java | 23 ++++++++++++----------
1 file changed, 13 insertions(+), 10 deletions(-)
diff --git a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreChunkPool.java b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreChunkPool.java
index eb43b43..763a574 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreChunkPool.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreChunkPool.java
@@ -77,6 +77,7 @@ public class MemStoreChunkPool {
private static final int statThreadPeriod = 60 * 5;
private AtomicLong createdChunkCount = new AtomicLong();
private AtomicLong reusedChunkCount = new AtomicLong();
+ private AtomicLong requestedChunkCount = new AtomicLong();
MemStoreChunkPool(Configuration conf, int chunkSize, int maxCount,
int initialCount) {
@@ -88,9 +89,9 @@ public class MemStoreChunkPool {
chunk.init();
reclaimedChunks.add(chunk);
}
- final String n = Thread.currentThread().getName();
+ createdChunkCount.set(initialCount);
scheduleThreadPool = Executors.newScheduledThreadPool(1,
- new ThreadFactoryBuilder().setNameFormat(n+"-MemStoreChunkPool Statistics")
+ new ThreadFactoryBuilder().setNameFormat("MemStoreChunkPool Statistics")
.setDaemon(true).build());
this.scheduleThreadPool.scheduleAtFixedRate(new StatisticsThread(this),
statThreadPeriod, statThreadPeriod, TimeUnit.SECONDS);
@@ -102,6 +103,7 @@ public class MemStoreChunkPool {
* @return a chunk
*/
Chunk getChunk() {
+ requestedChunkCount.incrementAndGet();
Chunk chunk = reclaimedChunks.poll();
if (chunk == null) {
chunk = new Chunk(chunkSize);
@@ -172,15 +174,16 @@ public class MemStoreChunkPool {
}
private void logStats() {
- if (!LOG.isDebugEnabled()) return;
- long created = createdChunkCount.get();
+ long total = createdChunkCount.get();
long reused = reusedChunkCount.get();
- long total = created + reused;
- LOG.debug("Stats: current pool size=" + reclaimedChunks.size()
- + ",created chunk count=" + created
- + ",reused chunk count=" + reused
- + ",reuseRatio=" + (total == 0 ? "0" : StringUtils.formatPercent(
- (float) reused / (float) total, 2)));
+ long available = reclaimedChunks.size();
+ long requested = requestedChunkCount.get();
+ LOG.info("Stats: chunk in pool=" + available
+ + ", chunk in use=" + (total - available)
+ + ", total chunk=" + total
+ + ", reused chunk=" + reused
+ + ", reuse ratio=" + (requested == 0 ? "0" : StringUtils.formatPercent(
+ (float) reused / (float) requested, 2)));
}
/**