[3/3] incubator-kudu git commit: KUDU-1324. Fix SEGV in catalog manager handling under-replicated tablet

[jd...@apache.org]

KUDU-1324. Fix SEGV in catalog manager handling under-replicated tablet

Commit 31278211f1934890e6835c9db164a7dea87d826a introduced some
new logging when starting the 'AsyncAddServer' task in the catalog
manager that tries to send the AddServer RPC for an under-replicated
tablet. However, this can SEGV in the case that the tablet does
not currently have an elected leader.

This crash can be triggered when restarting the master while a tablet
is under-replicated. When it comes back up, the master may
receive the report of the under-replicated tablet. When it tries
to run the AsyncAddServer task, there is no known leader yet (e.g.
because the leader has not yet sent its tablet report), and thus
the task fails immediately and deletes itself. Calling task->description()
then accesses the freed memory and crashes.

An earlier version of this fix tried to fix the issue by keeping a
scoped_refptr to the task. However, this isn't sufficient because
task->description() will crash if there is no known target tablet
server.

In order to fix this regression for the 0.7.0 release, this patch
takes the simplest approach of just changing the log message to
include less detail. A regression test will be included in a later
patch.

Change-Id: I62037fbaa910a1da476a0ac2075afdcdbc460dc8
Reviewed-on: http://gerrit.cloudera.org:8080/2060
Reviewed-by: Jean-Daniel Cryans
Tested-by: Kudu Jenkins


Project: http://git-wip-us.apache.org/repos/asf/incubator-kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kudu/commit/802d0e4c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kudu/tree/802d0e4c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kudu/diff/802d0e4c

Branch: refs/heads/master
Commit: 802d0e4c53f12b4392544ee10dfb530a25812d4f
Parents: b841512
Author: Todd Lipcon <to...@apache.org>
Authored: Thu Feb 4 18:19:12 2016 -0800
Committer: Jean-Daniel Cryans <jd...@gerrit.cloudera.org>
Committed: Sat Feb 6 18:40:14 2016 +0000

----------------------------------------------------------------------
 src/kudu/master/catalog_manager.cc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-kudu/blob/802d0e4c/src/kudu/master/catalog_manager.cc
----------------------------------------------------------------------
diff --git a/src/kudu/master/catalog_manager.cc b/src/kudu/master/catalog_manager.cc
index 000a6a5..b3c0468 100644
--- a/src/kudu/master/catalog_manager.cc
+++ b/src/kudu/master/catalog_manager.cc
@@ -2476,9 +2476,9 @@ void CatalogManager::SendAddServerRequest(const scoped_refptr<TabletInfo>& table
   tablet->table()->AddTask(task);
   WARN_NOT_OK(task->Run(), "Failed to send new AddServer request");
 
-  // Need to print this after Run() because that's where it picks the TS which description()
-  // needs.
-  LOG(INFO) << "Started AddServer task: " << task->description();
+  // We can't access 'task' here because it may delete itself inside Run() in the
+  // case that the tablet has no known leader.
+  LOG(INFO) << "Started AddServer task for tablet " << tablet->tablet_id();
 }
 
 void CatalogManager::ExtractTabletsToProcess(