You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by mp...@apache.org on 2017/02/08 22:42:53 UTC
kudu git commit: Add Google Breakpad support to Kudu
Repository: kudu
Updated Branches:
refs/heads/master 54f529dd4 -> a806ce003
Add Google Breakpad support to Kudu
Breakpad creates minidumps upon crash, which are small files that
include stack-based information for each thread. These do not include
heap information, so they complement, not replace, core files for
debugging purposes.
This patch enables Google Breakpad for the kudu-tserver and kudu-master
processes.
Breakpad is part of the crash-reporting infrastructure open-sourced by
Google. It is also used by Mozilla Firefox and Google Chrome.
When a process crashes, Breakpad writes a small dump file called a
minidump to disk. Since these files are typically only a few MB in size,
this allows users to share critical crash information with developers
for offline analysis.
For more information, see the breakpad getting started docs at
http://chromium.googlesource.com/breakpad/breakpad/+/master/docs/getting_started_with_breakpad.md
By default, on crash, minidump files are written to a subdirectory of
the log directory for a given Kudu daemon process. This was chosen for
the following reasons:
1. The minidump files are relatively small, potentially comparable in
size to log files, and the log directory is a place an administrator
would look in when debugging.
2. This convention is consistent with what Apache Impala (incubating)
does for its minidump files.
Changes in this patch:
* Add breakpad to thirdparty.
* Add breakpad source archive creation script (Breakpad does not do
releases).
* Add a "hack" to the breakpad thirdparty installation routine to
forcibly add a breakpad/ prefix to the breakpad header files. This
frees us from having to modify our include path when building.
* Pull in (heavily modified) util/minidump.{cc,h} from Apache Impala
(incubating). The modifications consist of removing use of
boost::filesystem, conformance to the Kudu code style guidelines, and
some significant refactoring.
* Enable breakpad support for the kudu-tserver and kudu-master
processes.
* By default, invoke previously-installed signal handler if installed.
* Handle SIGUSR1 by safely using sigwait() to create minidumps on demand.
* Add #ifdefs and CMake logic for Linux vs Apple.
* Add test for all the deadly signals to ensure we get both a stack
trace and a minidump, except for the case of SIGTERM, where we should
not get a minidump (by design).
* Change a few library unit tests that relied on SIGUSR1 to use another
signal for their testing, such as SIGHUP or SIGUSR2.
* Remove unused includes from mini_tablet_server.cc
* Clean up signal handling logic when forking a subprocess.
* Hide breakpad symbols from libkuduclient.so
* Install failure function that simply calls abort() instead of first
printing a stacktrace.
Change-Id: I495695cc38b75377f20b0497093a81ed5baa887f
Reviewed-on: http://gerrit.cloudera.org:8080/5620
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Kudu Jenkins
Reviewed-by: David Ribeiro Alves <dr...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/a806ce00
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/a806ce00
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/a806ce00
Branch: refs/heads/master
Commit: a806ce00347701062e43d0bd709ca4c97c2a7fc2
Parents: 54f529d
Author: Mike Percy <mp...@apache.org>
Authored: Wed Feb 8 13:49:11 2017 -0800
Committer: Mike Percy <mp...@apache.org>
Committed: Wed Feb 8 22:42:32 2017 +0000
----------------------------------------------------------------------
CMakeLists.txt | 9 +
build-support/jenkins/build-and-test.sh | 7 +-
cmake_modules/FindBreakpadClient.cmake | 36 ++
.../org/apache/kudu/client/MiniKuduCluster.java | 18 +-
src/kudu/client/client_samples-test.sh | 14 +-
src/kudu/client/symbols.map | 11 +
src/kudu/integration-tests/CMakeLists.txt | 4 +
.../integration-tests/external_mini_cluster.cc | 7 +
.../integration-tests/external_mini_cluster.h | 4 +
.../minidump_generation-itest.cc | 90 +++++
src/kudu/master/mini_master.cc | 6 +-
src/kudu/server/server_base.cc | 32 +-
src/kudu/server/server_base.h | 11 +-
src/kudu/tserver/mini_tablet_server.cc | 21 +-
src/kudu/util/CMakeLists.txt | 3 +
src/kudu/util/debug-util-test.cc | 16 +-
src/kudu/util/env_posix.cc | 2 +-
src/kudu/util/logging.cc | 4 +
src/kudu/util/minidump-test.cc | 144 +++++++
src/kudu/util/minidump.cc | 378 +++++++++++++++++++
src/kudu/util/minidump.h | 102 +++++
src/kudu/util/subprocess-test.cc | 2 +-
src/kudu/util/subprocess.cc | 79 ++--
src/kudu/util/test_main.cc | 6 +
thirdparty/build-definitions.sh | 26 ++
thirdparty/build-thirdparty.sh | 9 +
thirdparty/download-thirdparty.sh | 4 +
thirdparty/preflight.py | 1 +
thirdparty/scripts/make-breakpad-src-archive.sh | 61 +++
thirdparty/vars.sh | 6 +
30 files changed, 1028 insertions(+), 85 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/CMakeLists.txt
----------------------------------------------------------------------
diff --git a/CMakeLists.txt b/CMakeLists.txt
index c03c86e..3ee9666 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -919,6 +919,15 @@ ADD_THIRDPARTY_LIB(crcutil
STATIC_LIB "${CRCUTIL_STATIC_LIB}"
SHARED_LIB "${CRCUTIL_SHARED_LIB}")
+## breakpad
+if (NOT APPLE)
+ find_package(BreakpadClient REQUIRED)
+ include_directories(SYSTEM ${BREAKPAD_CLIENT_INCLUDE_DIR})
+ ADD_THIRDPARTY_LIB(breakpad_client
+ STATIC_LIB "${BREAKPAD_CLIENT_STATIC_LIB}"
+ SHARED_LIB "${BREAKPAD_CLIENT_SHARED_LIB}")
+endif()
+
## llvm
# Note that llvm has a unique cmake setup. See kudu/codegen/CMakeLists.txt
# for details.
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/build-support/jenkins/build-and-test.sh
----------------------------------------------------------------------
diff --git a/build-support/jenkins/build-and-test.sh b/build-support/jenkins/build-and-test.sh
index ab6bcbf..cb86b00 100755
--- a/build-support/jenkins/build-and-test.sh
+++ b/build-support/jenkins/build-and-test.sh
@@ -495,10 +495,9 @@ fi
if [ $EXIT_STATUS == 0 ]; then
TEST_TMPDIR_CONTENTS=$(ls $TEST_TMPDIR)
if [ -n "$TEST_TMPDIR_CONTENTS" ]; then
- echo "All tests passed, yet some left behind their test output:"
- for SUBDIR in $TEST_TMPDIR_CONTENTS; do
- echo $SUBDIR
- done
+ echo "All tests passed, yet some left behind their test output."
+ echo "TEST_TMPDIR: $TEST_TMPDIR"
+ find $TEST_TMPDIR -ls
EXIT_STATUS=1
fi
fi
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/cmake_modules/FindBreakpadClient.cmake
----------------------------------------------------------------------
diff --git a/cmake_modules/FindBreakpadClient.cmake b/cmake_modules/FindBreakpadClient.cmake
new file mode 100644
index 0000000..b79db5c
--- /dev/null
+++ b/cmake_modules/FindBreakpadClient.cmake
@@ -0,0 +1,36 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# - Find libbreakpad_client (exception_handler.h, libbreakpad_client.a)
+# This module defines
+# BREAKPAD_CLIENT_INCLUDE_DIR, directory containing headers
+# BREAKPAD_CLIENT_STATIC_LIB, path to libbreakpad_client's static library
+# BREAKPAD_CLIENT_SHARED_LIB, path to libbreakpad_client's shared library
+
+find_path(BREAKPAD_CLIENT_INCLUDE_DIR breakpad/client/linux/handler/exception_handler.h
+ NO_CMAKE_SYSTEM_PATH
+ NO_SYSTEM_ENVIRONMENT_PATH)
+find_library(BREAKPAD_CLIENT_STATIC_LIB libbreakpad_client.a
+ NO_CMAKE_SYSTEM_PATH
+ NO_SYSTEM_ENVIRONMENT_PATH)
+find_library(BREAKPAD_CLIENT_SHARED_LIB breakpad_client
+ NO_CMAKE_SYSTEM_PATH
+ NO_SYSTEM_ENVIRONMENT_PATH)
+
+include(FindPackageHandleStandardArgs)
+find_package_handle_standard_args(BREAKPAD_CLIENT REQUIRED_VARS
+ BREAKPAD_CLIENT_SHARED_LIB BREAKPAD_CLIENT_STATIC_LIB BREAKPAD_CLIENT_INCLUDE_DIR)
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
----------------------------------------------------------------------
diff --git a/java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java b/java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
index 3b9bb49..fa4fea4 100644
--- a/java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
+++ b/java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
@@ -159,12 +159,17 @@ public class MiniKuduCluster implements AutoCloseable {
for (int i = 0; i < numTservers; i++) {
int rpcPort = ports.get(i * 2);
tserverPorts.add(rpcPort);
- String dataDirPath = baseDirPath + "/ts-" + i + "-" + now;
+ String tsBaseDirPath = baseDirPath + "/ts-" + i + "-" + now;
+ new File(tsBaseDirPath).mkdir();
+ String logDirPath = tsBaseDirPath + "/logs";
+ new File(logDirPath).mkdir();
+ String dataDirPath = tsBaseDirPath + "/data";
String flagsPath = TestUtils.getFlagsPath();
List<String> commandLine = Lists.newArrayList(
TestUtils.findBinary("kudu-tserver"),
"--flagfile=" + flagsPath,
+ "--log_dir=" + logDirPath,
"--fs_wal_dir=" + dataDirPath,
"--fs_data_dirs=" + dataDirPath,
"--flush_threshold_mb=1",
@@ -189,7 +194,7 @@ public class MiniKuduCluster implements AutoCloseable {
// We made a temporary copy of the flags; delete them later.
pathsToDelete.add(flagsPath);
}
- pathsToDelete.add(dataDirPath);
+ pathsToDelete.add(tsBaseDirPath);
}
}
@@ -226,7 +231,11 @@ public class MiniKuduCluster implements AutoCloseable {
long now = System.currentTimeMillis();
for (int i = 0; i < numMasters; i++) {
int port = masterRpcPorts.get(i);
- String dataDirPath = baseDirPath + "/master-" + i + "-" + now;
+ String masterBaseDirPath = baseDirPath + "/master-" + i + "-" + now;
+ new File(masterBaseDirPath).mkdir();
+ String logDirPath = masterBaseDirPath + "/logs";
+ new File(logDirPath).mkdir();
+ String dataDirPath = masterBaseDirPath + "/data";
String flagsPath = TestUtils.getFlagsPath();
// The web port must be reserved in the call to findFreePorts above and specified
// to avoid the scenario where:
@@ -238,6 +247,7 @@ public class MiniKuduCluster implements AutoCloseable {
List<String> commandLine = Lists.newArrayList(
TestUtils.findBinary("kudu-master"),
"--flagfile=" + flagsPath,
+ "--log_dir=" + logDirPath,
"--fs_wal_dir=" + dataDirPath,
"--fs_data_dirs=" + dataDirPath,
"--webserver_interface=" + bindHost,
@@ -265,7 +275,7 @@ public class MiniKuduCluster implements AutoCloseable {
// We made a temporary copy of the flags; delete them later.
pathsToDelete.add(flagsPath);
}
- pathsToDelete.add(dataDirPath);
+ pathsToDelete.add(masterBaseDirPath);
}
return lastFreePort + 1;
}
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/client/client_samples-test.sh
----------------------------------------------------------------------
diff --git a/src/kudu/client/client_samples-test.sh b/src/kudu/client/client_samples-test.sh
index 1726f11..2abbe6e 100755
--- a/src/kudu/client/client_samples-test.sh
+++ b/src/kudu/client/client_samples-test.sh
@@ -105,21 +105,23 @@ export TMPDIR=${TMPDIR:-/tmp}
export TEST_TMPDIR=${TEST_TMPDIR:-$TMPDIR/kudutest-$UID}
mkdir -p $TEST_TMPDIR
BASE_DIR=$(mktemp -d $TEST_TMPDIR/client_samples-test.XXXXXXXX)
+mkdir -p "$BASE_DIR/master/logs"
$OUTPUT_DIR/kudu-master \
--unlock_experimental_flags \
--default_num_replicas=1 \
- --log_dir=$BASE_DIR \
- --fs_wal_dir=$BASE_DIR/master \
- --fs_data_dirs=$BASE_DIR/master \
+ --log_dir=$BASE_DIR/master/logs \
+ --fs_wal_dir=$BASE_DIR/master/wals \
+ --fs_data_dirs=$BASE_DIR/master/data \
--webserver_interface=localhost \
--webserver_port=0 \
--rpc_bind_addresses=$LOCALHOST_IP &
MASTER_PID=$!
+mkdir -p "$BASE_DIR/ts/logs"
$OUTPUT_DIR/kudu-tserver \
--unlock_experimental_flags \
- --log_dir=$BASE_DIR \
- --fs_wal_dir=$BASE_DIR/ts \
- --fs_data_dirs=$BASE_DIR/ts \
+ --log_dir=$BASE_DIR/ts/logs \
+ --fs_wal_dir=$BASE_DIR/ts/wals \
+ --fs_data_dirs=$BASE_DIR/ts/data \
--rpc_bind_addresses=$LOCALHOST_IP \
--local_ip_for_outbound_sockets=$LOCALHOST_IP \
--webserver_interface=localhost \
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/client/symbols.map
----------------------------------------------------------------------
diff --git a/src/kudu/client/symbols.map b/src/kudu/client/symbols.map
index fe11596..a58c9cd 100644
--- a/src/kudu/client/symbols.map
+++ b/src/kudu/client/symbols.map
@@ -23,6 +23,11 @@
# devtoolset
__cxa_throw_bad_array*;
+ # breakpad
+ ConvertUTF*;
+ isLegalUTF8Sequence;
+ my_*;
+
extern "C++" {
# glog, gflags, and protobuf
*google::*;
@@ -30,6 +35,12 @@
gflags_mutex_namespace::*;
glog_internal_namespace_::*;
+ # breakpad
+ google_breakpad::*;
+ logger::write*;
+ CreateGUID*;
+ GUIDToString*;
+
# crcutil
crcutil::*;
crcutil_interface::*;
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/integration-tests/CMakeLists.txt
----------------------------------------------------------------------
diff --git a/src/kudu/integration-tests/CMakeLists.txt b/src/kudu/integration-tests/CMakeLists.txt
index 7e7d16e..f3e9019 100644
--- a/src/kudu/integration-tests/CMakeLists.txt
+++ b/src/kudu/integration-tests/CMakeLists.txt
@@ -85,6 +85,10 @@ ADD_KUDU_TEST(ts_tablet_manager-itest)
ADD_KUDU_TEST(webserver-stress-itest)
ADD_KUDU_TEST(write_throttling-itest)
+if (NOT APPLE)
+ ADD_KUDU_TEST(minidump_generation-itest)
+endif()
+
# Tests that should not be run automatically by ctest.
ADD_KUDU_TEST_NO_CTEST(version_migration-test)
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/integration-tests/external_mini_cluster.cc
----------------------------------------------------------------------
diff --git a/src/kudu/integration-tests/external_mini_cluster.cc b/src/kudu/integration-tests/external_mini_cluster.cc
index ef4e287..57f2874 100644
--- a/src/kudu/integration-tests/external_mini_cluster.cc
+++ b/src/kudu/integration-tests/external_mini_cluster.cc
@@ -630,6 +630,9 @@ Status ExternalDaemon::StartProcess(const vector<string>& user_flags) {
// strong encryption in tests.
argv.push_back("--server_rsa_key_length_bits=512");
+ // Disable minidumps by default since many tests purposely inject faults.
+ argv.push_back("--enable_minidumps=false");
+
// Disable log redaction.
argv.push_back("--log_redact_user_data=false");
@@ -800,6 +803,10 @@ pid_t ExternalDaemon::pid() const {
return process_->pid();
}
+Subprocess* ExternalDaemon::process() const {
+ return process_.get();
+}
+
void ExternalDaemon::Shutdown() {
if (!process_) return;
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/integration-tests/external_mini_cluster.h
----------------------------------------------------------------------
diff --git a/src/kudu/integration-tests/external_mini_cluster.h b/src/kudu/integration-tests/external_mini_cluster.h
index 801a436..880fd0a 100644
--- a/src/kudu/integration-tests/external_mini_cluster.h
+++ b/src/kudu/integration-tests/external_mini_cluster.h
@@ -349,6 +349,10 @@ class ExternalDaemon : public RefCountedThreadSafe<ExternalDaemon> {
// Causes a CHECK failure if the process is not running.
pid_t pid() const;
+ // Return the pointer to the undelying Subprocess if it is set.
+ // Otherwise, returns nullptr.
+ Subprocess* process() const;
+
// Set the path of the executable to run as a daemon.
// Overrides the exe path specified in the constructor.
// The daemon must be shut down before calling this method.
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/integration-tests/minidump_generation-itest.cc
----------------------------------------------------------------------
diff --git a/src/kudu/integration-tests/minidump_generation-itest.cc b/src/kudu/integration-tests/minidump_generation-itest.cc
new file mode 100644
index 0000000..2abec12
--- /dev/null
+++ b/src/kudu/integration-tests/minidump_generation-itest.cc
@@ -0,0 +1,90 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include <string>
+#include <vector>
+
+#include <gflags/gflags.h>
+
+#include "kudu/integration-tests/external_mini_cluster-itest-base.h"
+#include "kudu/gutil/strings/substitute.h"
+#include "kudu/util/path_util.h"
+#include "kudu/util/subprocess.h"
+#include "kudu/util/test_util.h"
+
+DECLARE_string(minidump_path);
+
+using std::string;
+using std::vector;
+using strings::Substitute;
+
+namespace kudu {
+
+// Test the creation of minidumps upon process crash.
+class MinidumpGenerationITest : public ExternalMiniClusterITestBase {
+ protected:
+ void WaitForMinidumps(int expected, const string& dir);
+};
+
+void MinidumpGenerationITest::WaitForMinidumps(int expected, const string& dir) {
+ AssertEventually([&] {
+ vector<string> matches;
+ ASSERT_OK(env_->Glob(JoinPathSegments(dir, "*.dmp"), &matches));
+ ASSERT_EQ(expected, matches.size());
+ });
+}
+
+TEST_F(MinidumpGenerationITest, TestCreateMinidumpOnCrash) {
+ // Minidumps are disabled by default in the ExternalMiniCluster.
+ NO_FATALS(StartCluster({"--enable_minidumps"}, {"--enable_minidumps"}, 1));
+
+ // Test kudu-tserver.
+ ExternalTabletServer* ts = cluster_->tablet_server(0);
+ string dir = Substitute("$0/$1/$2", ts->log_dir(), "minidumps", "kudu-tserver");
+ ts->process()->Kill(SIGABRT);
+ NO_FATALS(WaitForMinidumps(1, dir));
+
+ // Test kudu-master.
+ ExternalMaster* master = cluster_->master();
+ dir = Substitute("$0/$1/$2", master->log_dir(), "minidumps", "kudu-master");
+ master->process()->Kill(SIGABRT);
+ NO_FATALS(WaitForMinidumps(1, dir));
+}
+
+TEST_F(MinidumpGenerationITest, TestCreateMinidumpOnSIGUSR1) {
+ // Minidumps are disabled by default in the ExternalMiniCluster.
+ NO_FATALS(StartCluster({"--enable_minidumps"}, {"--enable_minidumps"}, 1));
+
+ // Enable minidumps and ensure SIGUSR1 generates them.
+ ExternalTabletServer* ts = cluster_->tablet_server(0);
+ string dir = Substitute("$0/$1/$2", ts->log_dir(), "minidumps", "kudu-tserver");
+ ts->process()->Kill(SIGUSR1);
+ NO_FATALS(WaitForMinidumps(1, dir));
+ NO_FATALS(cluster_->AssertNoCrashes());
+ cluster_->Shutdown();
+
+ // Disable minidumps and ensure SIGUSR1 remains non-fatal.
+ ts->mutable_flags()->push_back("--enable_minidumps=false");
+ ASSERT_OK(env_->DeleteRecursively(dir));
+ ASSERT_OK(env_->CreateDir(dir));
+ ASSERT_OK(cluster_->Restart());
+ ts->process()->Kill(SIGUSR1);
+ NO_FATALS(cluster_->AssertNoCrashes());
+ NO_FATALS(WaitForMinidumps(0, dir)); // There should be no dumps.
+}
+
+} // namespace kudu
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/master/mini_master.cc
----------------------------------------------------------------------
diff --git a/src/kudu/master/mini_master.cc b/src/kudu/master/mini_master.cc
index 927bd24..f5f0f87 100644
--- a/src/kudu/master/mini_master.cc
+++ b/src/kudu/master/mini_master.cc
@@ -32,6 +32,7 @@
using strings::Substitute;
+DECLARE_bool(enable_minidumps);
DECLARE_bool(rpc_server_allow_ephemeral_ports);
namespace kudu {
@@ -41,7 +42,10 @@ MiniMaster::MiniMaster(Env* env, string fs_root, uint16_t rpc_port)
: running_(false),
env_(env),
fs_root_(std::move(fs_root)),
- rpc_port_(rpc_port) {}
+ rpc_port_(rpc_port) {
+ // Disable minidump handler (we allow only one per process).
+ FLAGS_enable_minidumps = false;
+}
MiniMaster::~MiniMaster() {
CHECK(!running_);
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/server/server_base.cc
----------------------------------------------------------------------
diff --git a/src/kudu/server/server_base.cc b/src/kudu/server/server_base.cc
index 088250f..9125fa1 100644
--- a/src/kudu/server/server_base.cc
+++ b/src/kudu/server/server_base.cc
@@ -52,6 +52,7 @@
#include "kudu/util/logging.h"
#include "kudu/util/mem_tracker.h"
#include "kudu/util/metrics.h"
+#include "kudu/util/minidump.h"
#include "kudu/util/monotime.h"
#include "kudu/util/net/net_util.h"
#include "kudu/util/net/sockaddr.h"
@@ -100,6 +101,7 @@ shared_ptr<MemTracker> CreateMemTrackerForServer() {
ServerBase::ServerBase(string name, const ServerBaseOptions& options,
const string& metric_namespace)
: name_(std::move(name)),
+ minidump_handler_(new MinidumpExceptionHandler()),
mem_tracker_(CreateMemTrackerForServer()),
metric_registry_(new MetricRegistry()),
metric_entity_(METRIC_ENTITY_server.Instantiate(metric_registry_.get(),
@@ -207,7 +209,7 @@ Status ServerBase::Init() {
RETURN_NOT_OK_PREPEND(StartMetricsLogging(), "Could not enable metrics logging");
result_tracker_->StartGCThread();
- RETURN_NOT_OK(StartExcessGlogDeleterThread());
+ RETURN_NOT_OK(StartExcessLogFileDeleterThread());
return Status::OK();
}
@@ -319,23 +321,27 @@ void ServerBase::MetricsLoggingThread() {
WARN_NOT_OK(log.Close(), "Unable to close metric log");
}
-Status ServerBase::StartExcessGlogDeleterThread() {
- if (FLAGS_logtostderr) {
- return Status::OK();
- }
+Status ServerBase::StartExcessLogFileDeleterThread() {
// Try synchronously deleting excess log files once at startup to make sure it
// works, then start a background thread to continue deleting them in the
- // future.
- RETURN_NOT_OK_PREPEND(DeleteExcessLogFiles(options_.env), "Unable to delete excess log files");
- return Thread::Create("server", "excess-glog-deleter", &ServerBase::ExcessGlogDeleterThread,
- this, &excess_glog_deleter_thread_);
+ // future. Same with minidumps.
+ if (!FLAGS_logtostderr) {
+ RETURN_NOT_OK_PREPEND(DeleteExcessLogFiles(options_.env),
+ "Unable to delete excess log files");
+ }
+ RETURN_NOT_OK_PREPEND(minidump_handler_->DeleteExcessMinidumpFiles(options_.env),
+ "Unable to delete excess minidump files");
+ return Thread::Create("server", "excess-log-deleter", &ServerBase::ExcessLogFileDeleterThread,
+ this, &excess_log_deleter_thread_);
}
-void ServerBase::ExcessGlogDeleterThread() {
- // How often to attempt to clean up excess glog files.
+void ServerBase::ExcessLogFileDeleterThread() {
+ // How often to attempt to clean up excess glog and minidump files.
const MonoDelta kWait = MonoDelta::FromSeconds(60);
while (!stop_background_threads_latch_.WaitUntil(MonoTime::Now() + kWait)) {
WARN_NOT_OK(DeleteExcessLogFiles(options_.env), "Unable to delete excess log files");
+ WARN_NOT_OK(minidump_handler_->DeleteExcessMinidumpFiles(options_.env),
+ "Unable to delete excess minidump files");
}
}
@@ -373,8 +379,8 @@ void ServerBase::Shutdown() {
if (metrics_logging_thread_) {
metrics_logging_thread_->Join();
}
- if (excess_glog_deleter_thread_) {
- excess_glog_deleter_thread_->Join();
+ if (excess_log_deleter_thread_) {
+ excess_log_deleter_thread_->Join();
}
web_server_->Stop();
rpc_server_->Shutdown();
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/server/server_base.h
----------------------------------------------------------------------
diff --git a/src/kudu/server/server_base.h b/src/kudu/server/server_base.h
index 57cadd2..4bd39b5 100644
--- a/src/kudu/server/server_base.h
+++ b/src/kudu/server/server_base.h
@@ -34,6 +34,7 @@ class FsManager;
class MemTracker;
class MetricEntity;
class MetricRegistry;
+class MinidumpExceptionHandler;
class NodeInstancePB;
class RpcServer;
class ScopedGLogMetrics;
@@ -97,7 +98,7 @@ class ServerBase {
protected:
ServerBase(std::string name, const ServerBaseOptions& options,
- const std::string& metrics_namespace);
+ const std::string& metric_namespace);
virtual ~ServerBase();
Status Init();
@@ -107,6 +108,7 @@ class ServerBase {
const std::string name_;
+ std::unique_ptr<MinidumpExceptionHandler> minidump_handler_;
std::shared_ptr<MemTracker> mem_tracker_;
gscoped_ptr<MetricRegistry> metric_registry_;
scoped_refptr<MetricEntity> metric_entity_;
@@ -135,13 +137,14 @@ class ServerBase {
void MetricsLoggingThread();
std::string FooterHtml() const;
- Status StartExcessGlogDeleterThread();
- void ExcessGlogDeleterThread();
+ // Start thread to remove excess glog and minidump files.
+ Status StartExcessLogFileDeleterThread();
+ void ExcessLogFileDeleterThread();
ServerBaseOptions options_;
scoped_refptr<Thread> metrics_logging_thread_;
- scoped_refptr<Thread> excess_glog_deleter_thread_;
+ scoped_refptr<Thread> excess_log_deleter_thread_;
CountDownLatch stop_background_threads_latch_;
gscoped_ptr<ScopedGLogMetrics> glog_metrics_;
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/tserver/mini_tablet_server.cc
----------------------------------------------------------------------
diff --git a/src/kudu/tserver/mini_tablet_server.cc b/src/kudu/tserver/mini_tablet_server.cc
index 8b617c0..f4cb331 100644
--- a/src/kudu/tserver/mini_tablet_server.cc
+++ b/src/kudu/tserver/mini_tablet_server.cc
@@ -19,38 +19,21 @@
#include <utility>
-#include <glog/logging.h>
-
-#include "kudu/common/schema.h"
-#include "kudu/consensus/consensus.h"
-#include "kudu/consensus/consensus.pb.h"
-#include "kudu/consensus/log.h"
-#include "kudu/consensus/log.pb.h"
#include "kudu/consensus/metadata.pb.h"
-#include "kudu/gutil/macros.h"
#include "kudu/gutil/strings/substitute.h"
-#include "kudu/server/rpc_server.h"
-#include "kudu/server/webserver.h"
-#include "kudu/tablet/tablet.h"
-#include "kudu/tablet/tablet_peer.h"
#include "kudu/tablet/tablet-test-util.h"
#include "kudu/tserver/tablet_server.h"
#include "kudu/tserver/ts_tablet_manager.h"
-#include "kudu/util/maintenance_manager.h"
#include "kudu/util/net/sockaddr.h"
#include "kudu/util/status.h"
using std::pair;
-using kudu::consensus::Consensus;
-using kudu::consensus::ConsensusOptions;
-using kudu::consensus::OpId;
using kudu::consensus::RaftPeerPB;
using kudu::consensus::RaftConfigPB;
-using kudu::log::Log;
-using kudu::log::LogOptions;
using strings::Substitute;
+DECLARE_bool(enable_minidumps);
DECLARE_bool(rpc_server_allow_ephemeral_ports);
namespace kudu {
@@ -60,6 +43,8 @@ MiniTabletServer::MiniTabletServer(const string& fs_root,
uint16_t rpc_port)
: started_(false) {
+ // Disable minidump handler (we allow only one per process).
+ FLAGS_enable_minidumps = false;
// Start RPC server on loopback.
FLAGS_rpc_server_allow_ephemeral_ports = true;
opts_.rpc_opts.rpc_bind_addresses = Substitute("127.0.0.1:$0", rpc_port);
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/util/CMakeLists.txt
----------------------------------------------------------------------
diff --git a/src/kudu/util/CMakeLists.txt b/src/kudu/util/CMakeLists.txt
index 4aecdcf..719af21 100644
--- a/src/kudu/util/CMakeLists.txt
+++ b/src/kudu/util/CMakeLists.txt
@@ -156,6 +156,7 @@ set(UTIL_SRCS
memory/overwrite.cc
mem_tracker.cc
metrics.cc
+ minidump.cc
monotime.cc
mutex.cc
net/dns_resolver.cc
@@ -223,6 +224,7 @@ set(UTIL_LIBS
if(NOT APPLE)
set(UTIL_LIBS
${UTIL_LIBS}
+ breakpad_client
dl
rt
vmem)
@@ -379,6 +381,7 @@ ADD_KUDU_TEST(url-coding-test)
ADD_KUDU_TEST(user-test)
if (NOT APPLE)
+ ADD_KUDU_TEST(minidump-test)
ADD_KUDU_TEST(pstack_watcher-test)
endif()
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/util/debug-util-test.cc
----------------------------------------------------------------------
diff --git a/src/kudu/util/debug-util-test.cc b/src/kudu/util/debug-util-test.cc
index df734ed..a07c9c2 100644
--- a/src/kudu/util/debug-util-test.cc
+++ b/src/kudu/util/debug-util-test.cc
@@ -98,11 +98,11 @@ TEST_F(DebugUtilTest, TestSignalStackTrace) {
// Test that we can change the signal and that the stack traces still work,
// on the new signal.
- ASSERT_FALSE(IsSignalHandlerRegistered(SIGUSR1));
- ASSERT_OK(SetStackTraceSignal(SIGUSR1));
+ ASSERT_FALSE(IsSignalHandlerRegistered(SIGHUP));
+ ASSERT_OK(SetStackTraceSignal(SIGHUP));
// Should now be registered.
- ASSERT_TRUE(IsSignalHandlerRegistered(SIGUSR1));
+ ASSERT_TRUE(IsSignalHandlerRegistered(SIGHUP));
// SIGUSR2 should be relinquished.
ASSERT_FALSE(IsSignalHandlerRegistered(SIGUSR2));
@@ -113,17 +113,17 @@ TEST_F(DebugUtilTest, TestSignalStackTrace) {
// Switch back to SIGUSR2 and ensure it changes back.
ASSERT_OK(SetStackTraceSignal(SIGUSR2));
ASSERT_TRUE(IsSignalHandlerRegistered(SIGUSR2));
- ASSERT_FALSE(IsSignalHandlerRegistered(SIGUSR1));
+ ASSERT_FALSE(IsSignalHandlerRegistered(SIGHUP));
// Stack traces should work using the new handler.
ASSERT_STR_CONTAINS(DumpThreadStack(t->tid()), "SleeperThread");
- // Register our own signal handler on SIGUSR1, and ensure that
+ // Register our own signal handler on SIGHUP, and ensure that
// we get a bad Status if we try to use it.
- signal(SIGUSR1, &fake_signal_handler);
- ASSERT_STR_CONTAINS(SetStackTraceSignal(SIGUSR1).ToString(),
+ signal(SIGHUP, &fake_signal_handler);
+ ASSERT_STR_CONTAINS(SetStackTraceSignal(SIGHUP).ToString(),
"unable to install signal handler");
- signal(SIGUSR1, SIG_IGN);
+ signal(SIGHUP, SIG_DFL);
// Stack traces should be disabled
ASSERT_STR_CONTAINS(DumpThreadStack(t->tid()), "unable to take thread stack");
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/util/env_posix.cc
----------------------------------------------------------------------
diff --git a/src/kudu/util/env_posix.cc b/src/kudu/util/env_posix.cc
index 3dff7d9..0a7b4bd 100644
--- a/src/kudu/util/env_posix.cc
+++ b/src/kudu/util/env_posix.cc
@@ -1243,7 +1243,7 @@ class PosixEnv : public Env {
ThreadRestrictions::AssertIOAllowed();
unique_ptr<char[], FreeDeleter> r(realpath(path.c_str(), nullptr));
if (!r) {
- return IOError(path, errno);
+ return IOError(Substitute("Unable to canonicalize $0", path), errno);
}
*result = string(r.get());
return Status::OK();
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/util/logging.cc
----------------------------------------------------------------------
diff --git a/src/kudu/util/logging.cc b/src/kudu/util/logging.cc
index 8332349..6cd56f2 100644
--- a/src/kudu/util/logging.cc
+++ b/src/kudu/util/logging.cc
@@ -40,6 +40,7 @@
#include "kudu/util/env.h"
#include "kudu/util/env_util.h"
#include "kudu/util/flag_tags.h"
+#include "kudu/util/minidump.h"
#include "kudu/util/status.h"
DEFINE_string(log_filename, "",
@@ -267,6 +268,9 @@ void InitGoogleLoggingSafe(const char* arg) {
// Sink logging: off.
initial_stderr_severity = FLAGS_stderrthreshold;
+ // For minidump support. Must be called before logging threads started.
+ CHECK_OK(BlockSigUSR1());
+
if (FLAGS_log_async) {
EnableAsyncLogging();
}
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/util/minidump-test.cc
----------------------------------------------------------------------
diff --git a/src/kudu/util/minidump-test.cc b/src/kudu/util/minidump-test.cc
new file mode 100644
index 0000000..2b39d97
--- /dev/null
+++ b/src/kudu/util/minidump-test.cc
@@ -0,0 +1,144 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include <signal.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <string>
+#include <vector>
+
+#include <gflags/gflags.h>
+
+#include "kudu/gutil/strings/substitute.h"
+#include "kudu/util/minidump.h"
+#include "kudu/util/path_util.h"
+#include "kudu/util/test_macros.h"
+#include "kudu/util/test_util.h"
+
+using std::string;
+using std::vector;
+
+DECLARE_bool(enable_minidumps);
+DECLARE_int32(max_minidumps);
+DECLARE_string(minidump_path);
+
+namespace kudu {
+
+class MinidumpDeathTest : public KuduTest {
+ protected:
+ void WaitForMinidumps(int expected, const string& dir);
+};
+
+void MinidumpDeathTest::WaitForMinidumps(int expected, const string& dir) {
+ AssertEventually([&] {
+ vector<string> matches;
+ ASSERT_OK(env_->Glob(JoinPathSegments(dir, "*.dmp"), &matches));
+ ASSERT_EQ(expected, matches.size());
+ });
+}
+
+// Test that registering the minidump exception handler results in creation of
+// minidump files on crash. Also test that deleting excess minidump files works
+// as expected.
+TEST_F(MinidumpDeathTest, TestRegisterAndDelete) {
+ FLAGS_enable_minidumps = true;
+ FLAGS_minidump_path = JoinPathSegments(test_dir_, "minidumps");
+ MinidumpExceptionHandler minidump_handler;
+ ASSERT_DEATH({
+ abort();
+ },
+ // Ensure that a stack trace is produced.
+ "kudu::MinidumpDeathTest_TestRegisterAndDelete_Test::TestBody()");
+
+ // Ensure that a minidump is produced.
+ string minidump_dir = minidump_handler.minidump_dir();
+ NO_FATALS(WaitForMinidumps(1, minidump_dir));
+
+ // Now create more minidumps so we can clean them up.
+ for (int num_dumps : {2, 3}) {
+ kill(getpid(), SIGUSR1);
+ NO_FATALS(WaitForMinidumps(num_dumps, minidump_dir));
+ }
+
+ FLAGS_max_minidumps = 2;
+ ASSERT_OK(minidump_handler.DeleteExcessMinidumpFiles(env_));
+ NO_FATALS(WaitForMinidumps(2, minidump_dir));
+}
+
+// Test that a CHECK() failure produces a stack trace and a minidump.
+TEST_F(MinidumpDeathTest, TestCheckStackTraceAndMinidump) {
+ FLAGS_enable_minidumps = true;
+ FLAGS_minidump_path = JoinPathSegments(test_dir_, "minidumps");
+ MinidumpExceptionHandler minidump_handler;
+ ASSERT_DEATH({
+ CHECK_EQ(1, 0);
+ },
+ // Ensure that a stack trace is produced.
+ "kudu::MinidumpDeathTest_TestCheckStackTraceAndMinidump_Test::TestBody()");
+
+ // Ensure that a minidump is produced.
+ string minidump_dir = minidump_handler.minidump_dir();
+ NO_FATALS(WaitForMinidumps(1, minidump_dir));
+}
+
+class MinidumpSignalDeathTest : public MinidumpDeathTest,
+ public ::testing::WithParamInterface<int> {
+};
+
+// Test that we get both a minidump and a stack trace for each supported signal.
+TEST_P(MinidumpSignalDeathTest, TestHaveMinidumpAndStackTrace) {
+ FLAGS_enable_minidumps = true;
+ int signal = GetParam();
+
+#if defined(ADDRESS_SANITIZER)
+ // ASAN appears to catch SIGBUS, SIGSEGV, and SIGFPE and the process is not killed.
+ if (signal == SIGBUS || signal == SIGSEGV || signal == SIGFPE) {
+ return;
+ }
+#endif
+
+#if defined(THREAD_SANITIZER)
+ // TSAN appears to catch SIGTERM and the process is not killed.
+ if (signal == SIGTERM) {
+ return;
+ }
+#endif
+
+ LOG(INFO) << "Testing signal: " << strsignal(signal);
+
+ FLAGS_minidump_path = JoinPathSegments(test_dir_, "minidumps");
+ MinidumpExceptionHandler minidump_handler;
+ ASSERT_DEATH({
+ kill(getpid(), signal);
+ },
+ // Ensure that a stack trace is produced.
+ "kudu::MinidumpSignalDeathTest_TestHaveMinidumpAndStackTrace_Test::TestBody()");
+
+ // Ensure that a mindump is produced, unless it's SIGTERM, which does not
+ // create a minidump.
+ int num_expected_minidumps = 1;
+ if (signal == SIGTERM) {
+ num_expected_minidumps = 0;
+ }
+ NO_FATALS(WaitForMinidumps(num_expected_minidumps, minidump_handler.minidump_dir()));
+}
+
+INSTANTIATE_TEST_CASE_P(DeadlySignals, MinidumpSignalDeathTest,
+ ::testing::Values(SIGABRT, SIGBUS, SIGSEGV, SIGILL, SIGFPE, SIGTERM));
+
+} // namespace kudu
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/util/minidump.cc
----------------------------------------------------------------------
diff --git a/src/kudu/util/minidump.cc b/src/kudu/util/minidump.cc
new file mode 100644
index 0000000..c088ae7
--- /dev/null
+++ b/src/kudu/util/minidump.cc
@@ -0,0 +1,378 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "kudu/util/minidump.h"
+
+#include <signal.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <atomic>
+#include <memory>
+#include <string>
+
+#if defined(__linux__)
+#include <breakpad/client/linux/handler/exception_handler.h>
+#include <breakpad/common/linux/linux_libc_support.h>
+#endif // defined(__linux__)
+
+#include <gflags/gflags.h>
+#include <glog/logging.h>
+
+#include "kudu/gutil/linux_syscall_support.h"
+#include "kudu/gutil/strings/human_readable.h"
+#include "kudu/util/errno.h"
+#include "kudu/util/env.h"
+#include "kudu/util/env_util.h"
+#include "kudu/util/flag_tags.h"
+#include "kudu/util/logging.h"
+#include "kudu/util/path_util.h"
+#include "kudu/util/status.h"
+
+using kudu::env_util::CreateDirIfMissing;
+using std::string;
+
+#if defined(__linux__)
+static constexpr bool kMinidumpPlatformSupported = true;
+#else
+static constexpr bool kMinidumpPlatformSupported = false;
+#endif // defined(__linux__)
+
+DECLARE_string(log_dir);
+
+DEFINE_bool(enable_minidumps, kMinidumpPlatformSupported,
+ "Whether to enable minidump generation upon process crash or SIGUSR1. "
+ "Currently only supported on Linux systems.");
+TAG_FLAG(enable_minidumps, advanced);
+TAG_FLAG(enable_minidumps, evolving);
+static bool ValidateMinidumpEnabled(const char* /*flagname*/, bool value) {
+ if (value && !kMinidumpPlatformSupported) {
+ return false; // NOLINT(*)
+ }
+ return true;
+}
+DEFINE_validator(enable_minidumps, &ValidateMinidumpEnabled);
+
+DEFINE_string(minidump_path, "minidumps", "Directory to write minidump files to. This "
+ "can be either an absolute path or a path relative to --log_dir. Each daemon will "
+ "create an additional sub-directory to prevent naming conflicts and to make it "
+ "easier to identify a crashing daemon. Minidump files contain crash-related "
+ "information in a compressed format. Minidumps will be written when a daemon exits "
+ "unexpectedly, for example on an unhandled exception or signal, or when a "
+ "SIGUSR1 signal is sent to the process. Cannot be set to an empty value.");
+TAG_FLAG(minidump_path, evolving);
+// The minidump path cannot be empty.
+static bool ValidateMinidumpPath(const char* /*flagname*/, const string& value) {
+ return !value.empty();
+}
+DEFINE_validator(minidump_path, &ValidateMinidumpPath);
+
+DEFINE_int32(max_minidumps, 9, "Maximum number of minidump files to keep per daemon. "
+ "Older files are removed first. Set to 0 to keep all minidump files.");
+TAG_FLAG(max_minidumps, evolving);
+
+DEFINE_int32(minidump_size_limit_hint_kb, 20480, "Size limit hint for minidump files in "
+ "KB. If a minidump exceeds this value, then breakpad will reduce the stack memory it "
+ "collects for each thread from 8KB to 2KB. However it will always include the full "
+ "stack memory for the first 20 threads, including the thread that crashed.");
+TAG_FLAG(minidump_size_limit_hint_kb, advanced);
+TAG_FLAG(minidump_size_limit_hint_kb, evolving);
+
+#if !defined(__linux__)
+namespace google_breakpad {
+// Define this as an empty class to avoid an undefined symbol error on Mac.
+class ExceptionHandler {
+ public:
+ ExceptionHandler() {}
+ ~ExceptionHandler() {}
+};
+} // namespace google_breakpad
+#endif // !defined(__linux__)
+
+namespace kudu {
+
+static sigset_t GetSigset(int signo) {
+ sigset_t signals;
+ CHECK_EQ(0, sigemptyset(&signals));
+ CHECK_EQ(0, sigaddset(&signals, signo));
+ return signals;
+}
+
+#if defined(__linux__)
+
+// Called by the exception handler before minidump is produced.
+// Minidump is only written if this returns true.
+static bool FilterCallback(void* /*context*/) {
+ return true;
+}
+
+// Write two null-terminated strings and a newline to both stdout and stderr.
+static void WriteLineStdoutStderr(const char* msg1, const char* msg2) {
+ // We use Breakpad's reimplementation of strlen(), called my_strlen(), from
+ // linux_libc_support.h to avoid calling into libc.
+ // A comment from linux_libc_support.h is reproduced here:
+ // "This header provides replacements for libc functions that we need. If we
+ // call the libc functions directly we risk crashing in the dynamic linker as
+ // it tries to resolve uncached PLT entries."
+ int msg1_len = my_strlen(msg1);
+ int msg2_len = my_strlen(msg2);
+
+ // We use sys_write() from linux_syscall_support.h here per the
+ // recommendation of the breakpad docs for the same reasons as above.
+ for (int fd : {STDOUT_FILENO, STDERR_FILENO}) {
+ sys_write(fd, msg1, msg1_len);
+ sys_write(fd, msg2, msg2_len);
+ sys_write(fd, "\n", 1);
+ }
+}
+
+// Callback for breakpad. It is called whenever a minidump file has been
+// written and should not be called directly. It logs the event before breakpad
+// crashes the process. Due to the process being in a failed state we write to
+// stdout/stderr and let the surrounding redirection make sure the output gets
+// logged. The calls might still fail in unknown scenarios as the process is in
+// a broken state. However we don't rely on them as the minidump file has been
+// written already.
+static bool DumpCallback(const google_breakpad::MinidumpDescriptor& descriptor,
+ void* context, bool succeeded) {
+
+ // Indicate whether a minidump file was written successfully. Write message
+ // to stdout/stderr, which will usually be captured in the INFO/ERROR log.
+ if (succeeded) {
+ WriteLineStdoutStderr("Wrote minidump to ", descriptor.path());
+ } else {
+ WriteLineStdoutStderr("Failed to write minidump to ", descriptor.path());
+ }
+
+ // If invoked by a user signal, return the actual success or failure of
+ // writing the minidump file so that we can print a user-friendly error
+ // message if writing the minidump fails.
+ bool is_user_signal = context != nullptr && *reinterpret_cast<bool*>(context);
+ if (is_user_signal) {
+ return succeeded;
+ }
+
+ // For crash signals. If we didn't want to invoke the previously-installed
+ // signal handler from glog, we would return the value received in
+ // 'succeeded' as described in the breakpad documentation. If this callback
+ // function returned true, breakpad would not invoke the previously-installed
+ // signal handler; instead, it would invoke the default signal handler, which
+ // would cause the process to crash immediately after writing the minidump.
+ //
+ // We make this callback always return false so that breakpad will invoke any
+ // previously-installed signal handler afterward. We want that to happen
+ // because the glog signal handlers print a helpful stacktrace on crash.
+ // That's convenient to have, because unlike a minidump, it doesn't need to
+ // be decoded to be useful for debugging.
+ return false;
+}
+
+// Failure function that simply calls abort().
+static void AbortFailureFunction() {
+ abort();
+}
+
+bool MinidumpExceptionHandler::WriteMinidump() {
+ bool user_signal = true;
+ return google_breakpad::ExceptionHandler::WriteMinidump(minidump_dir(),
+ &DumpCallback,
+ &user_signal);
+}
+
+Status MinidumpExceptionHandler::InitMinidumpExceptionHandler() {
+ minidump_dir_ = FLAGS_minidump_path;
+ if (minidump_dir_[0] != '/') {
+ minidump_dir_ = JoinPathSegments(FLAGS_log_dir, minidump_dir_);
+ }
+
+ // Create the first-level minidump directory.
+ Env* env = Env::Default();
+ RETURN_NOT_OK_PREPEND(CreateDirIfMissing(env, minidump_dir_),
+ "Error creating top-level minidump directory");
+
+ // Add the executable name to the path where minidumps will be written. This makes
+ // identification easier and prevents name collisions between the files.
+ // This is also consistent with how Impala organizes its minidump files.
+ const char* exe_name = gflags::ProgramInvocationShortName();
+ minidump_dir_ = JoinPathSegments(minidump_dir_, exe_name);
+
+ // Create the directory if it is not there. The minidump doesn't get written if there is
+ // no directory.
+ RETURN_NOT_OK_PREPEND(CreateDirIfMissing(env, minidump_dir_),
+ "Error creating minidump directory");
+
+ // Verify that the minidump directory really is a directory. We canonicalize
+ // in case it's a symlink to a directory.
+ string canonical_minidump_path;
+ RETURN_NOT_OK(env->Canonicalize(minidump_dir_, &canonical_minidump_path));
+ bool is_dir;
+ RETURN_NOT_OK(env->IsDirectory(canonical_minidump_path, &is_dir));
+ if (!is_dir) {
+ return Status::IOError("Unable to create minidump directory", canonical_minidump_path);
+ }
+
+ google_breakpad::MinidumpDescriptor desc(minidump_dir_);
+
+ // Limit filesize if configured.
+ if (FLAGS_minidump_size_limit_hint_kb > 0) {
+ size_t size_limit = 1024 * static_cast<int64_t>(FLAGS_minidump_size_limit_hint_kb);
+ LOG(INFO) << "Setting minidump size limit to "
+ << HumanReadableNumBytes::ToStringWithoutRounding(size_limit);
+ desc.set_size_limit(size_limit);
+ }
+
+ // If we don't uninstall the glog failure function when minidumps are enabled
+ // then we get two (2) stack traces printed from a LOG(FATAL) or CHECK(): one
+ // from the glog failure function and one from the glog signal handler. That
+ // is because we always return false in DumpCallback() in the non-user signal
+ // case.
+ google::InstallFailureFunction(&AbortFailureFunction);
+
+ breakpad_handler_.reset(
+ new google_breakpad::ExceptionHandler(desc, // Path to minidump directory.
+ FilterCallback, // Indicates whether to write the dump.
+ DumpCallback, // Output a log message when dumping.
+ nullptr, // Optional context for callbacks.
+ true, // Whether to install a crash handler.
+ -1)); // -1: Use in-process dump generation.
+
+ return Status::OK();
+}
+
+Status MinidumpExceptionHandler::RegisterMinidumpExceptionHandler() {
+ if (!FLAGS_enable_minidumps) return Status::OK();
+
+ // Ensure only one active instance is alive per process at any given time.
+ CHECK_EQ(0, MinidumpExceptionHandler::current_num_instances_.fetch_add(1));
+ RETURN_NOT_OK(InitMinidumpExceptionHandler());
+ RETURN_NOT_OK(StartUserSignalHandlerThread());
+ return Status::OK();
+}
+
+void MinidumpExceptionHandler::UnregisterMinidumpExceptionHandler() {
+ if (!FLAGS_enable_minidumps) return;
+
+ StopUserSignalHandlerThread();
+ CHECK_EQ(1, MinidumpExceptionHandler::current_num_instances_.fetch_sub(1));
+}
+
+Status MinidumpExceptionHandler::StartUserSignalHandlerThread() {
+ user_signal_handler_thread_running_.store(true, std::memory_order_relaxed);
+ return Thread::Create("minidump", "sigusr1-handler",
+ &MinidumpExceptionHandler::RunUserSignalHandlerThread,
+ this, &user_signal_handler_thread_);
+}
+
+void MinidumpExceptionHandler::StopUserSignalHandlerThread() {
+ user_signal_handler_thread_running_.store(false, std::memory_order_relaxed);
+ std::atomic_thread_fence(std::memory_order_release); // Store before signal.
+ // Send SIGUSR1 signal to thread, which will wake it up.
+ kill(getpid(), SIGUSR1);
+ user_signal_handler_thread_->Join();
+}
+
+void MinidumpExceptionHandler::RunUserSignalHandlerThread() {
+ sigset_t signals = GetSigset(SIGUSR1);
+ while (true) {
+ int signal;
+ int err = sigwait(&signals, &signal);
+ CHECK(err == 0) << "sigwait(): " << ErrnoToString(err) << ": " << err;
+ CHECK_EQ(SIGUSR1, signal);
+ if (!user_signal_handler_thread_running_.load(std::memory_order_relaxed)) {
+ // Exit thread if we are shutting down.
+ return;
+ }
+ if (!WriteMinidump()) {
+ LOG(WARNING) << "Received USR1 signal but failed to write minidump";
+ }
+ }
+}
+
+#else // defined(__linux__)
+
+// At the time of writing, we don't support breakpad on Mac so we just stub out
+// all the methods defined in the header file.
+
+Status MinidumpExceptionHandler::InitMinidumpExceptionHandler() {
+ return Status::OK();
+}
+
+// No-op on non-Linux platforms.
+Status MinidumpExceptionHandler::RegisterMinidumpExceptionHandler() {
+ return Status::OK();
+}
+
+void MinidumpExceptionHandler::UnregisterMinidumpExceptionHandler() {
+}
+
+bool MinidumpExceptionHandler::WriteMinidump() {
+ return true;
+}
+
+Status MinidumpExceptionHandler::StartUserSignalHandlerThread() {
+ return Status::OK();
+}
+
+void MinidumpExceptionHandler::StopUserSignalHandlerThread() {
+}
+
+void MinidumpExceptionHandler::RunUserSignalHandlerThread() {
+}
+
+#endif // defined(__linux__)
+
+std::atomic<int> MinidumpExceptionHandler::current_num_instances_;
+
+MinidumpExceptionHandler::MinidumpExceptionHandler() {
+ CHECK_OK(RegisterMinidumpExceptionHandler());
+}
+
+MinidumpExceptionHandler::~MinidumpExceptionHandler() {
+ UnregisterMinidumpExceptionHandler();
+}
+
+Status MinidumpExceptionHandler::DeleteExcessMinidumpFiles(Env* env) {
+ // Do not delete minidump files if minidumps are disabled.
+ if (!FLAGS_enable_minidumps) return Status::OK();
+
+ int32_t max_minidumps = FLAGS_max_minidumps;
+ // Disable rotation if set to 0 or less.
+ if (max_minidumps <= 0) return Status::OK();
+
+ // Minidump filenames are created by breakpad in the following format, for example:
+ // 7b57915b-ee6a-dbc5-21e59491-5c60a2cf.dmp.
+ string pattern = JoinPathSegments(minidump_dir(), "*.dmp");
+
+ // Use mtime to determine which minidumps to delete. While this could
+ // potentially be ambiguous if many minidumps were created in quick
+ // succession, users can always increase 'FLAGS_max_minidumps' if desired
+ // in order to work around the problem.
+ return env_util::DeleteExcessFilesByPattern(env, pattern, max_minidumps);
+}
+
+string MinidumpExceptionHandler::minidump_dir() const {
+ return minidump_dir_;
+}
+
+Status BlockSigUSR1() {
+ sigset_t signals = GetSigset(SIGUSR1);
+ int ret = pthread_sigmask(SIG_BLOCK, &signals, nullptr);
+ if (ret == 0) return Status::OK();
+ return Status::InvalidArgument("pthread_sigmask", ErrnoToString(ret), ret);
+}
+
+} // namespace kudu
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/util/minidump.h
----------------------------------------------------------------------
diff --git a/src/kudu/util/minidump.h b/src/kudu/util/minidump.h
new file mode 100644
index 0000000..58702d7
--- /dev/null
+++ b/src/kudu/util/minidump.h
@@ -0,0 +1,102 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <atomic>
+#include <memory>
+#include <string>
+
+#include "kudu/gutil/ref_counted.h"
+#include "kudu/util/locks.h"
+#include "kudu/util/thread.h"
+
+namespace google_breakpad {
+class ExceptionHandler;
+} // namespace google_breakpad
+
+namespace kudu {
+
+class Env;
+class Status;
+
+// While an instance of this class is in scope, a Breakpad minidump handler
+// will generate a minidump for the current program if it crashes or if it
+// received a USR1 signal. This class must be instantiated after initializing
+// the gflags library. When used in conjuction with glog, or other signal
+// handling facilities, this class must be invoked after installing those
+// signal handlers.
+//
+// The BlockSigUSR1() function should be called before spawning any threads in
+// order to block the USR1 signal from crashing the process. This class relies
+// on that signal being blocked by all threads in order to safely generate
+// minidumps in response to the USR1 signal.
+//
+// Only one instance of this class may be instantiated at a time.
+//
+// For more information on Google Breakpad, see its documentation at:
+// http://chromium.googlesource.com/breakpad/breakpad/+/master/docs/getting_started_with_breakpad.md
+class MinidumpExceptionHandler {
+ public:
+ MinidumpExceptionHandler();
+ ~MinidumpExceptionHandler();
+
+ // Write a minidump immediately. Can be used to generate a minidump
+ // independently of a crash. Should not be called from a signal handler or a
+ // crash context because it uses the heap.
+ bool WriteMinidump();
+
+ // Deletes excess minidump files beyond the configured max of
+ // 'FLAGS_max_minidumps'. Uses the file's modified time to determine recency.
+ // Does nothing if 'FLAGS_enabled_minidumps' is false.
+ Status DeleteExcessMinidumpFiles(Env* env);
+
+ // Get the path to the directory that will be used for writing minidumps.
+ std::string minidump_dir() const;
+
+ private:
+ Status InitMinidumpExceptionHandler();
+ Status RegisterMinidumpExceptionHandler();
+ void UnregisterMinidumpExceptionHandler();
+
+ Status StartUserSignalHandlerThread();
+ void StopUserSignalHandlerThread();
+ void RunUserSignalHandlerThread();
+
+ // The number of instnaces of this class that are currently in existence.
+ // We keep this counter in order to force a crash if more than one is running
+ // at a time, as a sanity check.
+ static std::atomic<int> current_num_instances_;
+
+ std::atomic<bool> user_signal_handler_thread_running_ ATTRIBUTE_UNUSED; // Unused in macOS build.
+ scoped_refptr<Thread> user_signal_handler_thread_;
+
+ // Breakpad ExceptionHandler. It registers its own signal handlers to write
+ // minidump files during process crashes, but can also be used to write
+ // minidumps directly.
+ std::unique_ptr<google_breakpad::ExceptionHandler> breakpad_handler_;
+
+ // Directory in which we store our minidumps.
+ std::string minidump_dir_;
+};
+
+// Block SIGUSR1 from threads handling it.
+// This should be called by the process before spawning any threads so that a
+// USR1 signal will cause a minidump to be generated instead of a crash.
+Status BlockSigUSR1();
+
+} // namespace kudu
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/util/subprocess-test.cc
----------------------------------------------------------------------
diff --git a/src/kudu/util/subprocess-test.cc b/src/kudu/util/subprocess-test.cc
index 86238a8..515d862 100644
--- a/src/kudu/util/subprocess-test.cc
+++ b/src/kudu/util/subprocess-test.cc
@@ -207,7 +207,7 @@ TEST_F(SubprocessTest, TestGetExitStatusSignaled) {
SIGABRT,
SIGKILL,
SIGTERM,
- SIGUSR1,
+ SIGUSR2,
};
for (auto signum : kSignals) {
Subprocess p("/bin/cat", { "cat" });
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/util/subprocess.cc
----------------------------------------------------------------------
diff --git a/src/kudu/util/subprocess.cc b/src/kudu/util/subprocess.cc
index 0c8436c..f69abe8 100644
--- a/src/kudu/util/subprocess.cc
+++ b/src/kudu/util/subprocess.cc
@@ -72,23 +72,40 @@ static const char* kProcSelfFd =
#if defined(__linux__)
#define READDIR readdir64
#define DIRENT dirent64
+typedef sighandler_t SignalHandlerCallback;
#else
#define READDIR readdir
#define DIRENT dirent
+typedef sig_t SignalHandlerCallback;
#endif
-void DisableSigPipe() {
+// Convenience wrapper for sigaction().
+void SetSignalHandler(int signal, SignalHandlerCallback handler) {
struct sigaction act;
-
- act.sa_handler = SIG_IGN;
+ act.sa_handler = handler;
sigemptyset(&act.sa_mask);
act.sa_flags = 0;
- PCHECK(sigaction(SIGPIPE, &act, nullptr) == 0);
+ PCHECK(sigaction(signal, &act, nullptr) == 0);
+}
+
+void IgnoreSigPipe() {
+ SetSignalHandler(SIGPIPE, SIG_IGN);
+}
+
+void ResetSigPipeHandlerToDefault() {
+ SetSignalHandler(SIGPIPE, SIG_DFL);
}
-void EnsureSigPipeDisabled() {
+void EnsureSigPipeIgnored() {
static GoogleOnceType once = GOOGLE_ONCE_INIT;
- GoogleOnceInit(&once, &DisableSigPipe);
+ GoogleOnceInit(&once, &IgnoreSigPipe);
+}
+
+// We unblock all signal masks since they are inherited.
+void ResetAllSignalMasksToUnblocked() {
+ sigset_t signals;
+ PCHECK(sigfillset(&signals) == 0);
+ PCHECK(sigprocmask(SIG_UNBLOCK, &signals, nullptr) == 0);
}
// Since opendir() calls malloc(), this must be called before fork().
@@ -322,7 +339,7 @@ Status Subprocess::Start() {
if (argv_.empty()) {
return Status::InvalidArgument("argv must have at least one elem");
}
- EnsureSigPipeDisabled();
+ EnsureSigPipeIgnored();
vector<char*> argv_ptrs;
for (const string& arg : argv_) {
@@ -375,36 +392,48 @@ Status Subprocess::Start() {
if (fd_state_[STDIN_FILENO] == PIPED) {
PCHECK(dup2(child_stdin[0], STDIN_FILENO) == STDIN_FILENO);
}
+
// stdout
switch (fd_state_[STDOUT_FILENO]) {
- case PIPED: {
- PCHECK(dup2(child_stdout[1], STDOUT_FILENO) == STDOUT_FILENO);
- break;
- }
- case DISABLED: {
- RedirectToDevNull(STDOUT_FILENO);
- break;
- }
- default: break;
+ case PIPED: {
+ PCHECK(dup2(child_stdout[1], STDOUT_FILENO) == STDOUT_FILENO);
+ break;
+ }
+ case DISABLED: {
+ RedirectToDevNull(STDOUT_FILENO);
+ break;
+ }
+ default:
+ break;
}
+
// stderr
switch (fd_state_[STDERR_FILENO]) {
- case PIPED: {
- PCHECK(dup2(child_stderr[1], STDERR_FILENO) == STDERR_FILENO);
- break;
- }
- case DISABLED: {
- RedirectToDevNull(STDERR_FILENO);
- break;
- }
- default: break;
+ case PIPED: {
+ PCHECK(dup2(child_stderr[1], STDERR_FILENO) == STDERR_FILENO);
+ break;
+ }
+ case DISABLED: {
+ RedirectToDevNull(STDERR_FILENO);
+ break;
+ }
+ default:
+ break;
}
+
// Close the read side of the sync pipe;
// the write side should be closed upon execvp().
PCHECK(close(sync_pipe[0]) == 0);
CloseNonStandardFDs(fd_dir);
+ // Ensure we are not ignoring or blocking signals in the child process.
+ ResetAllSignalMasksToUnblocked();
+ // Reset SIGPIPE to its default disposition because we previously set it to
+ // SIG_IGN via EnsureSigPipeIgnored(). At the time of writing, we don't
+ // explicitly ignore any other signals in Kudu.
+ ResetSigPipeHandlerToDefault();
+
// Set the environment for the subprocess. This is more portable than
// using execvpe(), which doesn't exist on OS X. We rely on the 'p'
// variant of exec to do $PATH searching if the executable specified
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/src/kudu/util/test_main.cc
----------------------------------------------------------------------
diff --git a/src/kudu/util/test_main.cc b/src/kudu/util/test_main.cc
index 2935b24..74fe4e9 100644
--- a/src/kudu/util/test_main.cc
+++ b/src/kudu/util/test_main.cc
@@ -24,6 +24,7 @@
#include "kudu/util/pstack_watcher.h"
#include "kudu/util/flags.h"
+#include "kudu/util/minidump.h"
#include "kudu/util/status.h"
#include "kudu/util/test_util.h"
@@ -65,6 +66,11 @@ static void StartStressThreads() {
int main(int argc, char **argv) {
google::InstallFailureSignalHandler();
+
+ // We don't use InitGoogleLoggingSafe() because gtest initializes glog, so we
+ // need to block SIGUSR1 explicitly in order to test minidump generation.
+ CHECK_OK(kudu::BlockSigUSR1());
+
// InitGoogleTest() must precede ParseCommandLineFlags(), as the former
// removes gtest-related flags from argv that would trip up the latter.
::testing::InitGoogleTest(&argc, argv);
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/thirdparty/build-definitions.sh
----------------------------------------------------------------------
diff --git a/thirdparty/build-definitions.sh b/thirdparty/build-definitions.sh
index 66c062e..270d0e6 100644
--- a/thirdparty/build-definitions.sh
+++ b/thirdparty/build-definitions.sh
@@ -567,6 +567,32 @@ build_crcutil() {
popd
}
+build_breakpad() {
+ BREAKPAD_BDIR=$TP_BUILD_DIR/$BREAKPAD_NAME$MODE_SUFFIX
+ mkdir -p $BREAKPAD_BDIR
+ pushd $BREAKPAD_BDIR
+
+ CFLAGS="$EXTRA_CFLAGS" \
+ CXXFLAGS="$EXTRA_CXXFLAGS" \
+ LDFLAGS="$EXTRA_LDFLAGS" \
+ LIBS="$EXTRA_LIBS" \
+ $BREAKPAD_SOURCE/configure --prefix=$PREFIX
+ make -j$PARALLEL $EXTRA_MAKEFLAGS install
+
+ # Here we run a script to munge breakpad #include statements in-place. Sadly,
+ # breakpad does not prefix its header file paths. Until we can solve that
+ # issue upstream we use this "hack" to add breakpad/ as a prefix for all the
+ # breakpad-related includes in the breakpad header files ourselves. See also
+ # https://bugs.chromium.org/p/google-breakpad/issues/detail?id=721
+ local BREAKPAD_INCLUDE_DIR=$PREFIX/include/breakpad
+ pushd "$BREAKPAD_INCLUDE_DIR"
+ find . -type f | xargs grep -l "#include" | \
+ xargs perl -p -i -e '@pre = qw(client common google_breakpad processor third_party); for $p (@pre) { s{#include "$p/}{#include "breakpad/$p/}; }'
+ popd
+
+ popd
+}
+
build_cpplint() {
# Copy cpplint tool into bin directory
cp $GSG_SOURCE/cpplint/cpplint.py $PREFIX/bin/cpplint.py
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/thirdparty/build-thirdparty.sh
----------------------------------------------------------------------
diff --git a/thirdparty/build-thirdparty.sh b/thirdparty/build-thirdparty.sh
index 4ce4271..04b52a5 100755
--- a/thirdparty/build-thirdparty.sh
+++ b/thirdparty/build-thirdparty.sh
@@ -91,6 +91,7 @@ else
"trace-viewer") F_TRACE_VIEWER=1 ;;
"nvml") F_NVML=1 ;;
"boost") F_BOOST=1 ;;
+ "breakpad") F_BREAKPAD=1 ;;
*) echo "Unknown module: $arg"; exit 1 ;;
esac
done
@@ -289,6 +290,10 @@ if [ -n "$F_UNINSTRUMENTED" -o -n "$F_BOOST" ]; then
build_boost normal
fi
+if [ -n "$F_UNINSTRUMENTED" -o -n "$F_BREAKPAD" ]; then
+ build_breakpad
+fi
+
restore_env
# If we're on MacOs best to exit here, otherwise single dependency builds will try to
@@ -452,6 +457,10 @@ if [ -n "$F_TSAN" -o -n "$F_BOOST" ]; then
build_boost tsan
fi
+if [ -n "$F_TSAN" -o -n "$F_BREAKPAD" ]; then
+ build_breakpad
+fi
+
restore_env
finish
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/thirdparty/download-thirdparty.sh
----------------------------------------------------------------------
diff --git a/thirdparty/download-thirdparty.sh b/thirdparty/download-thirdparty.sh
index a061bff..252405e 100755
--- a/thirdparty/download-thirdparty.sh
+++ b/thirdparty/download-thirdparty.sh
@@ -298,5 +298,9 @@ if needs_openssl_workaround && [ ! -d "$OPENSSL_WORKAROUND_DIR" ] ; then
$TP_DIR/install-openssl-el6-workaround.sh
fi
+if [ ! -d "$BREAKPAD_SOURCE" ]; then
+ fetch_and_expand breakpad-${BREAKPAD_VERSION}.tar.gz
+fi
+
echo "---------------"
echo "Thirdparty dependencies downloaded successfully"
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/thirdparty/preflight.py
----------------------------------------------------------------------
diff --git a/thirdparty/preflight.py b/thirdparty/preflight.py
index 88e0a83..4e071f3 100755
--- a/thirdparty/preflight.py
+++ b/thirdparty/preflight.py
@@ -43,6 +43,7 @@ REQUIRED_TOOLS = [
"git",
"make",
"patch",
+ "perl",
"pkg-config",
"rsync",
"unzip",
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/thirdparty/scripts/make-breakpad-src-archive.sh
----------------------------------------------------------------------
diff --git a/thirdparty/scripts/make-breakpad-src-archive.sh b/thirdparty/scripts/make-breakpad-src-archive.sh
new file mode 100755
index 0000000..3fb98f1
--- /dev/null
+++ b/thirdparty/scripts/make-breakpad-src-archive.sh
@@ -0,0 +1,61 @@
+#!/bin/bash -e
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+################################################################################
+# Script to make a source archive from the Breakpad project, which does not run
+# releases.
+#
+# We export only checked-in files, without the git metadata, and construct
+# the expected directory structure to compile it. Then we tar it up.
+
+BREAKPAD_REPO=https://chromium.googlesource.com/breakpad/breakpad
+LSS_REPO=https://chromium.googlesource.com/linux-syscall-support
+LSS_REVISION=3f6478ac95edf86cd3da300c2c0d34a438f5dbeb
+
+TOPDIR=breakpad-repos
+
+rm -rf "$TOPDIR"
+mkdir $TOPDIR
+pushd $TOPDIR
+
+BREAKPAD_DIR=breakpad.git
+git clone $BREAKPAD_REPO $BREAKPAD_DIR
+pushd $BREAKPAD_DIR
+
+REVISION=${1:-$(git rev-parse HEAD)}
+NAME=breakpad-$REVISION
+
+git checkout $REVISION
+# Export the checked-in files, without .git metadata, into "../$NAME/".
+git archive --format=tar --prefix=$NAME/ $REVISION | (cd .. && tar xf -)
+popd
+
+LSS_DIR=lss.git
+git clone $LSS_REPO $LSS_DIR
+pushd $LSS_DIR
+# Export into the thirdparty directory under "../$NAME/".
+git archive --format=tar --prefix=$NAME/src/third_party/lss/ $LSS_REVISION | (cd .. && tar xf -)
+popd
+
+FILENAME="$NAME.tar.gz"
+tar czvf "$FILENAME" "$NAME/"
+mv "$FILENAME" ..
+popd
+echo "Archive created at $(pwd)/$FILENAME"
+
+exit 0
http://git-wip-us.apache.org/repos/asf/kudu/blob/a806ce00/thirdparty/vars.sh
----------------------------------------------------------------------
diff --git a/thirdparty/vars.sh b/thirdparty/vars.sh
index e44abce..d653afd 100644
--- a/thirdparty/vars.sh
+++ b/thirdparty/vars.sh
@@ -169,3 +169,9 @@ BOOST_NAME=boost_$BOOST_VERSION
BOOST_SOURCE=$TP_SOURCE_DIR/$BOOST_NAME
OPENSSL_WORKAROUND_DIR="$TP_DIR/installed/openssl-el6-workaround"
+
+# The breakpad source artifact is created using the script found in
+# scripts/make-breakpad-src-archive.sh
+BREAKPAD_VERSION=f78d953511606348173911ae0b62572ebec1bbc4
+BREAKPAD_NAME=breakpad-$BREAKPAD_VERSION
+BREAKPAD_SOURCE=$TP_SOURCE_DIR/$BREAKPAD_NAME