kudu git commit: pstack_watcher: set a limit on backtrace depth

[to...@apache.org]

Repository: kudu
Updated Branches:
  refs/heads/master 5bdb69409 -> 6ed469079


pstack_watcher: set a limit on backtrace depth

On RHEL 6.8, pstack_watcher-test times out fairly reliably, with a 'gdb'
process left spinning 100% CPU.

Some googling found a report of a similar spinning GDB while taking a
backtrace[1]. This report suggests setting a backtrace limit --
apparently the spinning is due to gdb getting in some infinite loop
trying to walk the stack with insufficient/incorrect debug info.

This patch sets such a limit. I verified on a box which was reliably
reproducing the timeout that it is now fixed.

[1] https://bugs.launchpad.net/ubuntu/+source/gdb/+bug/434168

Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
Reviewed-on: http://gerrit.cloudera.org:8080/6658
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Reviewed-by: Mike Percy <mp...@apache.org>
Tested-by: Kudu Jenkins


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/6ed46907
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/6ed46907
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/6ed46907

Branch: refs/heads/master
Commit: 6ed469079605b44d7bfe300b8ee6d4167a844c86
Parents: 5bdb694
Author: Todd Lipcon <to...@apache.org>
Authored: Mon Apr 17 16:17:40 2017 -0700
Committer: Todd Lipcon <to...@apache.org>
Committed: Mon Apr 17 23:54:38 2017 +0000

----------------------------------------------------------------------
 src/kudu/util/pstack_watcher.cc | 10 ++++++++++
 1 file changed, 10 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/6ed46907/src/kudu/util/pstack_watcher.cc
----------------------------------------------------------------------
diff --git a/src/kudu/util/pstack_watcher.cc b/src/kudu/util/pstack_watcher.cc
index ec24a11..dbb22a9 100644
--- a/src/kudu/util/pstack_watcher.cc
+++ b/src/kudu/util/pstack_watcher.cc
@@ -132,9 +132,19 @@ Status PstackWatcher::RunGdbStackDump(pid_t pid, int flags) {
   string prog("gdb");
   vector<string> argv;
   argv.push_back(prog);
+  // Don't print introductory version/copyright messages.
   argv.push_back("-quiet");
+  // Exit after processing all of the commands below.
   argv.push_back("-batch");
+  // Don't run commands from .gdbinit
   argv.push_back("-nx");
+  // On RHEL6 and older Ubuntu, we occasionally would see gdb spin forever
+  // trying to collect backtraces. Setting a backtrace limit is a reasonable
+  // workaround, since we don't really expect >100-deep stacks anyway.
+  //
+  // See https://bugs.launchpad.net/ubuntu/+source/gdb/+bug/434168
+  argv.push_back("-ex");
+  argv.push_back("set backtrace limit 100");
   argv.push_back("-ex");
   argv.push_back("set print pretty on");
   argv.push_back("-ex");