You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by to...@apache.org on 2016/02/04 00:27:51 UTC

[4/4] incubator-kudu git commit: build: some coverage build fixes

build: some coverage build fixes

There's only really one important fix here (the addition of SOURCE_ROOT);
the rest is stuff I noticed along the way. We don't need the removal of
gcda/gcno files as build-and-test.sh now blows away the entire build root
before running.

Various tests log the following:

  LLVM ERROR: Program used external function 'llvm_gcda_summary_info'
  which could not be resolved!

In some cases it's fatal, while in others (tests using external clusters)
it's benign. I suspect it's to do with codegen and coverage instrumentation.

Many tests also output lines like:

  profiling:/home/adar/Source/kudu/build/coverage/src/kudu/rpc/CMakeFiles/krpc.dir/connection.cc.gcda:Merge mismatch for function 0

I don't know what this means or what the effect is.

Finally, mt-tablet-test and full_stack-insert-scan-test deadlock in codegen.
I think this has to do with the external function errors above, as the stack
traces show some sort of error while LLVM tries to resolve external symbols:

  Thread 2 (Thread 0x7f8033bba700 (LWP 10131)):
  #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  #1  0x0000000001b61a8a in kudu::ConditionVariable::Wait (this=0x5228200) at /home/adar/Source/kudu/src/kudu/util/condition_variable.cc:66
  #2  0x0000000001c6d040 in kudu::ThreadPool::Shutdown (this=0x5228160) at /home/adar/Source/kudu/src/kudu/util/threadpool.cc:151
  #3  0x0000000000ef13e7 in kudu::codegen::CompilationManager::Shutdown () at /home/adar/Source/kudu/src/kudu/codegen/compilation_manager.cc:145
  #4  0x00007f8040c0ed32 in __run_exit_handlers (status=1, listp=0x7f8040f99698 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true) at exit.c:82
  #5  0x00007f8040c0ed85 in __GI_exit (status=<optimized out>) at exit.c:104
  #6  0x0000000001885e30 in llvm::report_fatal_error(llvm::Twine const&, bool) ()
  #7  0x0000000000f60916 in llvm::RuntimeDyldImpl::resolveExternalSymbols() ()
  #8  0x0000000000f6459e in llvm::RuntimeDyldImpl::resolveRelocations() ()
  #9  0x0000000000f4b6f7 in llvm::MCJIT::finalizeLoadedModules() ()
  #10 0x0000000000f4bad6 in llvm::MCJIT::finalizeObject() ()
  #11 0x0000000000f114f2 in kudu::codegen::ModuleBuilder::Compile (this=0x7f8033bb9690, out=0x7f8033bb95f0) at /home/adar/Source/kudu/src/kudu/codegen/module_builder.cc:274
  #12 0x0000000000ef6bed in kudu::codegen::RowProjectorFunctions::Create (base_schema=..., projection=..., out=0x7f8033bb9950, tm=0x7f8033bb9768) at /home/adar/Source/kudu/src/kudu/codegen/row_projector.cc:294
  #13 0x0000000000f0a0f6 in kudu::codegen::CodeGenerator::CompileRowProjector (this=0x5266cf0, base=..., proj=..., out=0x7f8033bb9950) at /home/adar/Source/kudu/src/kudu/codegen/code_generator.cc:218
  #14 0x0000000000ef0370 in kudu::codegen::(anonymous namespace)::CompilationTask::RunWithStatus (this=0x5472000) at /home/adar/Source/kudu/src/kudu/codegen/compilation_manager.cc:103
  #15 0x0000000000eef9d1 in kudu::codegen::(anonymous namespace)::CompilationTask::Run (this=0x5472000) at /home/adar/Source/kudu/src/kudu/codegen/compilation_manager.cc:85
  #16 0x0000000001c6efa6 in kudu::ThreadPool::DispatchThread (this=0x5228160, permanent=false) at /home/adar/Source/kudu/src/kudu/util/threadpool.cc:316
  ...

  Thread 1 (Thread 0x7f8042269800 (LWP 10102)):
  #0  0x00007f8041c5c8ab in __waitpid (pid=26562, stat_loc=0x7fff53de97ec, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:40
  #1  0x0000000001c4b6ef in kudu::Subprocess::DoWait (this=0x7fff53de9850, ret=0x7fff53de97ec, options=0) at /home/adar/Source/kudu/src/kudu/util/subprocess.cc:342
  #2  0x0000000001bcd39e in kudu::Subprocess::Wait (this=0x7fff53de9850, ret=0x7fff53de97ec) at /home/adar/Source/kudu/src/kudu/util/subprocess.h:75
  #3  0x0000000001bcca8f in kudu::PstackWatcher::RunStackDump (prog="gdb", argv=std::vector of length 12, capacity 16 = {...}) at /home/adar/Source/kudu/src/kudu/util/pstack_watcher.cc:177
  #4  0x0000000001bcbb24 in kudu::PstackWatcher::RunGdbStackDump (pid=10102, flags=0) at /home/adar/Source/kudu/src/kudu/util/pstack_watcher.cc:154
  #5  0x0000000001bcacaf in kudu::PstackWatcher::DumpPidStacks (pid=10102, flags=0) at /home/adar/Source/kudu/src/kudu/util/pstack_watcher.cc:114
  #6  0x0000000001bcabd6 in kudu::PstackWatcher::DumpStacks (flags=0) at /home/adar/Source/kudu/src/kudu/util/pstack_watcher.cc:107
  #7  0x0000000000d348b0 in KillTestOnTimeout (signum=14) at /home/adar/Source/kudu/src/kudu/util/test_main.cc:72
  #8  <signal handler called>
  #9  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  #10 0x0000000001b61a8a in kudu::ConditionVariable::Wait (this=0x52281c0) at /home/adar/Source/kudu/src/kudu/util/condition_variable.cc:66
  #11 0x0000000001c6e072 in kudu::ThreadPool::Wait (this=0x5228160) at /home/adar/Source/kudu/src/kudu/util/threadpool.cc:231
  #12 0x0000000000ef1379 in kudu::codegen::CompilationManager::Wait (this=0x5266cf0) at /home/adar/Source/kudu/src/kudu/codegen/compilation_manager.cc:141
  #13 0x0000000000ab5b30 in kudu::tablet::MultiThreadedTabletTest<kudu::tablet::StringKeyTestSetup>::SetUp (this=0x525f200) at /home/adar/Source/kudu/src/kudu/tablet/mt-tablet-test.cc:80

We've deadlocked the codegen thread pool because one its threads invokes the
atexit() handler, which calls back into the pool and waits on
a condition variable that only it can signal. It would be nice to at least
make this a fatal error rather than a deadlock (i.e. check in Shutdown()
that the caller isn't a thread belonging to the pool).

I've also reconfigured the job to handle the build directory changes. I'll
be monitoring it to see if it gets back on track.

Change-Id: Ief0bfc3fe55e4643be2338c8a54a852fc936a261
Reviewed-on: http://gerrit.cloudera.org:8080/2011
Tested-by: Adar Dembo <ad...@cloudera.com>
Reviewed-by: Todd Lipcon <to...@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/incubator-kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kudu/commit/bd492726
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kudu/tree/bd492726
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kudu/diff/bd492726

Branch: refs/heads/master
Commit: bd4927265e3e56722e5f2b9a41f6a3dc450c9b9b
Parents: affa0e4
Author: Adar Dembo <ad...@cloudera.com>
Authored: Tue Feb 2 20:30:58 2016 -0800
Committer: Todd Lipcon <to...@apache.org>
Committed: Wed Feb 3 23:02:38 2016 +0000

----------------------------------------------------------------------
 build-support/jenkins/build-and-test.sh | 4 +---
 src/kudu/client/CMakeLists.txt          | 9 ++++++---
 2 files changed, 7 insertions(+), 6 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-kudu/blob/bd492726/build-support/jenkins/build-and-test.sh
----------------------------------------------------------------------
diff --git a/build-support/jenkins/build-and-test.sh b/build-support/jenkins/build-and-test.sh
index c31a9c2..837a91b 100755
--- a/build-support/jenkins/build-and-test.sh
+++ b/build-support/jenkins/build-and-test.sh
@@ -181,8 +181,6 @@ elif [ "$BUILD_TYPE" = "COVERAGE" ]; then
   DO_COVERAGE=1
   BUILD_TYPE=debug
   $SOURCE_ROOT/build-support/enable_devtoolset.sh "$THIRDPARTY_BIN/cmake -DKUDU_GENERATE_COVERAGE=1 $SOURCE_ROOT"
-  # Reset coverage info from previous runs
-  find src -name \*.gcda -o -name \*.gcno -exec rm {} \;
 elif [ "$BUILD_TYPE" = "LINT" ]; then
   # Create empty test logs or else Jenkins fails to archive artifacts, which
   # results in the build failing.
@@ -292,7 +290,7 @@ if [ "$DO_COVERAGE" == "1" ]; then
   echo
   echo Generating coverage report...
   echo ------------------------------------------------------------
-  if ! ./thirdparty/gcovr-3.0/scripts/gcovr -r $SOURCE_ROOT --xml \
+  if ! $SOURCE_ROOT/thirdparty/gcovr-3.0/scripts/gcovr -r $SOURCE_ROOT --xml \
       > $BUILD_ROOT/coverage.xml ; then
     EXIT_STATUS=1
     FAILURES="$FAILURES"$'Coverage report failed\n'

http://git-wip-us.apache.org/repos/asf/incubator-kudu/blob/bd492726/src/kudu/client/CMakeLists.txt
----------------------------------------------------------------------
diff --git a/src/kudu/client/CMakeLists.txt b/src/kudu/client/CMakeLists.txt
index e5533f4..4ebae41 100644
--- a/src/kudu/client/CMakeLists.txt
+++ b/src/kudu/client/CMakeLists.txt
@@ -220,9 +220,12 @@ target_link_libraries(kudu_client_test_util
 
 # Tests
 
-if (NOT APPLE)
-  # The OS X system compiler does not support source symbol maps, so the client
-  # leaks internal symbols globally.
+# The OS X system compiler does not support source symbol maps, so the client
+# leaks internal symbols globally.
+#
+# Coverage builds insert gcov-related symbols into the client library. We
+# don't ship such builds, so there's no point in checking symbol visibility.
+if (NOT APPLE AND NOT "${KUDU_GENERATE_COVERAGE}")
   ADD_KUDU_TEST(client_symbol-test.sh LABELS no_dist_test)
 endif()