You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by al...@apache.org on 2019/02/05 22:41:01 UTC

[kudu] branch master updated (73cda9f -> 2b9b579)

This is an automated email from the ASF dual-hosted git repository.

alexey pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git.


    from 73cda9f  [docker] Optimize kudu image size
     new 1048738  Reduce "Failed to heartbeat..." log spam
     new 2b9b579  [tools] KUDU-2688 rebalancer should not fix RF=2 violations

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 src/kudu/tools/placement_policy_util-test.cc | 45 ++++++++++++++++++++++++++++
 src/kudu/tools/placement_policy_util.cc      | 12 ++++++++
 src/kudu/tserver/heartbeater.cc              |  5 ++--
 3 files changed, 60 insertions(+), 2 deletions(-)


[kudu] 01/02: Reduce "Failed to heartbeat..." log spam

Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit 10487386a15dd1882c4bbe81876e2bb7eb1b272c
Author: Will Berkeley <wd...@gmail.org>
AuthorDate: Tue Feb 5 13:34:53 2019 -0800

    Reduce "Failed to heartbeat..." log spam
    
    Tablet servers heartbeat to the master at 1Hz by default, and previously
    would log a message every time a heartbeat failed. This was a little
    much, as it's not unusual for a server out of contact with a master to
    fail heartbeats for quite some time. This patch lowers the frequency to
    once per minute, and also enhances the error message to include the
    number of failed heartbeats since the last successful heartbeat.
    
    Change-Id: Ia48e844ebeb565343e55dbceae1674c34ea8ed5b
    Reviewed-on: http://gerrit.cloudera.org:8080/12374
    Reviewed-by: Andrew Wong <aw...@cloudera.com>
    Tested-by: Kudu Jenkins
---
 src/kudu/tserver/heartbeater.cc | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/src/kudu/tserver/heartbeater.cc b/src/kudu/tserver/heartbeater.cc
index 2d226ba..b9cb0a8 100644
--- a/src/kudu/tserver/heartbeater.cc
+++ b/src/kudu/tserver/heartbeater.cc
@@ -583,8 +583,9 @@ void Heartbeater::Thread::RunThread() {
     const auto& s = DoHeartbeat(&error, &error_status);
     if (!s.ok()) {
       const auto& err_msg = s.ToString();
-      LOG(WARNING) << Substitute("Failed to heartbeat to $0: $1",
-                                 master_address_.ToString(), err_msg);
+      KLOG_EVERY_N_SECS(WARNING, 60)
+          << Substitute("Failed to heartbeat to $0 ($1 consecutive failures): $2",
+                        master_address_.ToString(), consecutive_failed_heartbeats_, err_msg);
       consecutive_failed_heartbeats_++;
       // If we encountered a network error (e.g., connection
       // refused), try reconnecting.


[kudu] 02/02: [tools] KUDU-2688 rebalancer should not fix RF=2 violations

Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit 2b9b579bbde4956f8bd4f4e2c3dd08a49b9b8bc0
Author: Alexey Serbin <al...@apache.org>
AuthorDate: Tue Feb 5 13:52:58 2019 -0800

    [tools] KUDU-2688 rebalancer should not fix RF=2 violations
    
    This patch addresses KUDU-2688.  Prior to this patch, the LA rebalancing
    logic would suggest moves to reinstate the placement policy various
    distributions of RF=2 tablet's replicas.  As a result, the rebalancer
    looped indefinitely trying to implement the suggested replica movements.
    However, RF=2 is a special case: any replica distribution of a tablet
    with RF=2 is a violations of the placement policy constraints and it's
    impossible to fix that, regardless of the number of locations
    in a Kudu cluster.
    
    With this patch, the rebalancer no longer tries to correct non-fixable
    violations of the placement policy for RF=2.
    
    Change-Id: I3afc7f3a228368e8646b46914ec7c31234a6aa85
    Reviewed-on: http://gerrit.cloudera.org:8080/12375
    Reviewed-by: Will Berkeley <wd...@gmail.com>
    Tested-by: Kudu Jenkins
---
 src/kudu/tools/placement_policy_util-test.cc | 45 ++++++++++++++++++++++++++++
 src/kudu/tools/placement_policy_util.cc      | 12 ++++++++
 2 files changed, 57 insertions(+)

diff --git a/src/kudu/tools/placement_policy_util-test.cc b/src/kudu/tools/placement_policy_util-test.cc
index cccd611..6eca2fe 100644
--- a/src/kudu/tools/placement_policy_util-test.cc
+++ b/src/kudu/tools/placement_policy_util-test.cc
@@ -545,6 +545,51 @@ TEST_F(ClusterLocationTest, PlacementPolicyViolationsEvenRFEdgeCases) {
       {}
     },
     {
+      // Three locations, RF=2.
+      {
+        { "T0", 2, { "t0", "t1", "t2", } },
+      },
+      {
+        { "L0", { "A", "B", } },
+        { "L1", { "C", "D", } },
+        { "L2", { "E", "F", } },
+      },
+      {
+        { "A", { "t0", } }, { "B", { "t0", } },
+        { "C", { "t1", } }, { "D", { "t2", } },
+        { "E", { "t1", } }, { "F", { "t2", } },
+      },
+      {
+        { "t0", "L0", 2 },
+        { "t1", "L2", 1 },
+        { "t2", "L2", 1 },
+      },
+      {}
+    },
+    {
+      // Four locations, RF=2.
+      {
+        { "T0", 2, { "t0", "t1", } },
+      },
+      {
+        { "L0", { "A", } },
+        { "L1", { "B", } },
+        { "L2", { "C", } },
+        { "L3", { "D", } },
+      },
+      {
+        { "A", { "t0", } },
+        { "B", { "t0", } },
+        { "C", { "t1", } },
+        { "D", { "t1", } },
+      },
+      {
+        { "t0", "L1", 1 },
+        { "t1", "L3", 1 },
+      },
+      {}
+    },
+    {
       // Two locations, RF=2 and RF=4.
       {
         { "T0", 2, { "t0", } },
diff --git a/src/kudu/tools/placement_policy_util.cc b/src/kudu/tools/placement_policy_util.cc
index f5ab790..48456f7 100644
--- a/src/kudu/tools/placement_policy_util.cc
+++ b/src/kudu/tools/placement_policy_util.cc
@@ -75,6 +75,18 @@ Status FindBestReplicaToReplace(
   const auto& table_id = FindOrDie(tablets_info.tablet_to_table_id, tablet_id);
   const auto& table_info = FindOrDie(tablets_info.tables_info, table_id);
 
+  // The replication factor of 2 is a special case: any possible replica
+  // distribution is a violation of placement policy, and it's impossible
+  // to fix it regardless of number of locations in the cluster.
+  if (table_info.replication_factor == 2) {
+    return Status::ConfigurationError(Substitute(
+        "tablet $0 (table name '$1'): replica distribution cannot conform "
+        "with the placement policy constraints since its replication "
+        "factor is $2",
+        tablet_id, table_info.name,
+        table_info.replication_factor));
+  }
+
   // There are a few edge cases which are most likely to occur, so let's have
   // a special error message for those. In these cases there are too few
   // locations relative to the replication factor, so it's impossible to find