You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by al...@apache.org on 2024/07/31 23:43:45 UTC

(kudu) branch master updated (4116941c0 -> a1cc227df)

This is an automated email from the ASF dual-hosted git repository.

alexey pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git


    from 4116941c0 KUDU-3591 Fix the flaky test FsManagerTestBase.TestAddRemoveDataDirsFuzz
     new 834db150f [tools] fix compilation warning
     new a1cc227df [cmake] work around of Gradle-related issues

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 CMakeLists.txt                      | 20 ++++++++++++++++++--
 src/kudu/tools/tool_action_table.cc |  4 ++--
 2 files changed, 20 insertions(+), 4 deletions(-)


(kudu) 01/02: [tools] fix compilation warning

Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit 834db150fe9907156cfc5ca6686879e471beb838
Author: Alexey Serbin <al...@apache.org>
AuthorDate: Wed Jul 31 11:12:41 2024 -0700

    [tools] fix compilation warning
    
    This patch addresses compilation warnings like below (GCC 4.8.5):
    
      src/kudu/tools/tool_action_table.cc:334:24: warning: extra tokens at end of #undef directive
           #undef GET_PROPERTY()
                              ^
      src/kudu/tools/tool_action_table.cc:335:28: warning: extra tokens at end of #undef directive
           #undef GET_NUM_PROPERTY()
    
    This is a follow-up to d91d5c95dab38770890cac6a30be63f80eb82fec.
    
    Change-Id: I2484aebb5c3334549e9b04957121f556d1120933
    Reviewed-on: http://gerrit.cloudera.org:8080/21625
    Tested-by: Alexey Serbin <al...@apache.org>
    Reviewed-by: Abhishek Chennaka <ac...@cloudera.com>
---
 src/kudu/tools/tool_action_table.cc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/kudu/tools/tool_action_table.cc b/src/kudu/tools/tool_action_table.cc
index 21b8c04e8..5c985998b 100644
--- a/src/kudu/tools/tool_action_table.cc
+++ b/src/kudu/tools/tool_action_table.cc
@@ -331,8 +331,8 @@ class TableLister {
             {"live_row_count", GET_NUM_PROPERTY(live_row_count)},
         };
 
-    #undef GET_PROPERTY()
-    #undef GET_NUM_PROPERTY()
+    #undef GET_PROPERTY
+    #undef GET_NUM_PROPERTY
 
     for (const auto& tinfo : tables_info) {
       vector<string> values;


(kudu) 02/02: [cmake] work around of Gradle-related issues

Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit a1cc227df5d5dc7586d3971135643320682b2709
Author: Alexey Serbin <al...@apache.org>
AuthorDate: Wed Jul 31 11:58:55 2024 -0700

    [cmake] work around of Gradle-related issues
    
    Once Gradle started being invoked to build kudu-subprocess and kudu-hms
    JARs as dependency for the C++ server-side of the project, it's become
    an issue to get 100% success rate when building it from source.  With
    the recent Gradle upgrade the build success rate has reduced further,
    it seems.
    
    I decided to address that by adding a few extra flags to the Gradle's
    invocations under cmake when building corresponding targets.  Probably,
    there is a better way of addressing that (say, finding the root cause
    of the issues, reporting, and fixing them), but this simple solution
    works well for me.  Also, this solution doesn't affect the build times
    of the project (even when using ccache) since the targets built with
    Gradle constitute a small part of the rest of the targets that are built
    concurrently in a very robust manner.
    
    Prior to this patch, Gradle would error out on follow-up invocations
    after the build process has been interrupted, not being able to access
    its build cache with an error like below:
    
    --------------------------------------------------
      FAILURE: Build failed with an exception.
    
      * What went wrong:
      Execution failed for task ':kudu-hive:compileJava'.
      > Could not create service of type DefaultGeneralCompileCaches using GradleScopeCompileServices.createGeneralCompileCaches().
         > Cannot lock Java compile cache (...) as it has already been locked by this process.
    --------------------------------------------------
    
    Prior to this patch, building the project would fail sometimes when
    invoking 'cmake -jN' (where N > 1) with errors like below while Gradle
    was generating kudu-subprocess.jar and hms-plugin.jar concurrently:
    
    --------------------------------------------------
      ...
      [  4%] Generating ../../../bin/kudu-subprocess.jar
      ...
      [  6%] Generating ../../../bin/hms-plugin.jar
      ...
      java.lang.ArrayIndexOutOfBoundsException: 6
              at shadow.org.objectweb.asm.ClassReader.readShort(ClassReader.java:3573)
      ...
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
              at java.lang.Thread.run(Thread.java:750)
    
      FAILURE: Build failed with an exception.
    
      * What went wrong:
      Execution failed for task ':kudu-client:shadowJar'.
      > 6
      ...
      BUILD FAILED in 1m 0s
      make[2]: *** [bin/hms-plugin.jar] Error 1
      make[1]: *** [src/kudu/hms/CMakeFiles/hms_plugin_jar.dir/all] Error 2
      make[1]: *** Waiting for unfinished jobs....
      java.lang.ArrayIndexOutOfBoundsException: 6
              at shadow.org.objectweb.asm.ClassReader.readShort(ClassReader.java:3573)
      ...
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
    java:1149)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
    .java:624)
            at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnabl
    e.run(ThreadFactoryImpl.java:56)
            at java.lang.Thread.run(Thread.java:750)
    
      FAILURE: Build failed with an exception.
    
      * What went wrong:
      Execution failed for task ':kudu-subprocess:shadowJar'.
      > 6
      ...
      BUILD FAILED in 1m 3s
      make[2]: *** [bin/kudu-subprocess.jar] Error 1
      make[1]: *** [src/kudu/subprocess/CMakeFiles/subprocess_jar.dir/all] Error 2
      make: *** [all] Error 2
    --------------------------------------------------
    
    Change-Id: Ib7d1637854233592ede5388bfc2287df96ec9865
    Reviewed-on: http://gerrit.cloudera.org:8080/21626
    Reviewed-by: Abhishek Chennaka <ac...@cloudera.com>
    Tested-by: Alexey Serbin <al...@apache.org>
---
 CMakeLists.txt | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 0f6201447..3ebdd1b1a 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -109,8 +109,24 @@ set(JAVA_DIR ${CMAKE_CURRENT_SOURCE_DIR}/java)
 # This solves the problem of escaped spaces in the EXTRA_GRADLE_FLAGS environment variable.
 list(APPEND GRADLE_FLAGS $ENV{EXTRA_GRADLE_FLAGS})
 separate_arguments(GRADLE_FLAGS)
-# We always want Gradle to use the plain console and quiet flag when called from cmake.
-list(APPEND GRADLE_FLAGS --quiet --console=plain)
+# When invoking Gradle:
+#   * Log errors only, reducing the verbosity of the Gradle's output
+#   * Make the output better fit for plain consoles, avoiding garbled logs
+#   * Avoid building projects in parallel: when the kudu-subprocess.jar and
+#     the hms-plugin.jar targets are built in parallel, it often ends up with
+#     java.lang.ArrayIndexOutOfBoundsException while building shadowJar
+#     for corresponding Java dependencies, and while it's not yet clear whether
+#     the issue is in shadowJar implementation or elsewhere, it's quite simple
+#     to address the issue by not building multiple Gradle targets concurrently
+#   * Disable the build cache: sometimes, once a build process is interrupted,
+#     follow-up Gradle invocations end up with an error like below, and while
+#     it's not yet clear whether the issue is in not properly invoking Gradle
+#     or it's a bug in signal handling elsewhere, it's quite simple to address
+#     the issue by not using the Gradle's build cache
+#
+#       Cannot lock Java compile cache (...) as it has already been locked by this process.
+#
+list(APPEND GRADLE_FLAGS --quiet --console=plain --no-parallel --no-build-cache)
 list(REMOVE_DUPLICATES GRADLE_FLAGS)
 
 # Initialize Gradle to ensure the wrapper is downloaded.