You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by al...@apache.org on 2024/07/31 23:43:45 UTC
(kudu) branch master updated (4116941c0 -> a1cc227df)
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
from 4116941c0 KUDU-3591 Fix the flaky test FsManagerTestBase.TestAddRemoveDataDirsFuzz
new 834db150f [tools] fix compilation warning
new a1cc227df [cmake] work around of Gradle-related issues
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CMakeLists.txt | 20 ++++++++++++++++++--
src/kudu/tools/tool_action_table.cc | 4 ++--
2 files changed, 20 insertions(+), 4 deletions(-)
(kudu) 01/02: [tools] fix compilation warning
Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 834db150fe9907156cfc5ca6686879e471beb838
Author: Alexey Serbin <al...@apache.org>
AuthorDate: Wed Jul 31 11:12:41 2024 -0700
[tools] fix compilation warning
This patch addresses compilation warnings like below (GCC 4.8.5):
src/kudu/tools/tool_action_table.cc:334:24: warning: extra tokens at end of #undef directive
#undef GET_PROPERTY()
^
src/kudu/tools/tool_action_table.cc:335:28: warning: extra tokens at end of #undef directive
#undef GET_NUM_PROPERTY()
This is a follow-up to d91d5c95dab38770890cac6a30be63f80eb82fec.
Change-Id: I2484aebb5c3334549e9b04957121f556d1120933
Reviewed-on: http://gerrit.cloudera.org:8080/21625
Tested-by: Alexey Serbin <al...@apache.org>
Reviewed-by: Abhishek Chennaka <ac...@cloudera.com>
---
src/kudu/tools/tool_action_table.cc | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/kudu/tools/tool_action_table.cc b/src/kudu/tools/tool_action_table.cc
index 21b8c04e8..5c985998b 100644
--- a/src/kudu/tools/tool_action_table.cc
+++ b/src/kudu/tools/tool_action_table.cc
@@ -331,8 +331,8 @@ class TableLister {
{"live_row_count", GET_NUM_PROPERTY(live_row_count)},
};
- #undef GET_PROPERTY()
- #undef GET_NUM_PROPERTY()
+ #undef GET_PROPERTY
+ #undef GET_NUM_PROPERTY
for (const auto& tinfo : tables_info) {
vector<string> values;
(kudu) 02/02: [cmake] work around of Gradle-related issues
Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit a1cc227df5d5dc7586d3971135643320682b2709
Author: Alexey Serbin <al...@apache.org>
AuthorDate: Wed Jul 31 11:58:55 2024 -0700
[cmake] work around of Gradle-related issues
Once Gradle started being invoked to build kudu-subprocess and kudu-hms
JARs as dependency for the C++ server-side of the project, it's become
an issue to get 100% success rate when building it from source. With
the recent Gradle upgrade the build success rate has reduced further,
it seems.
I decided to address that by adding a few extra flags to the Gradle's
invocations under cmake when building corresponding targets. Probably,
there is a better way of addressing that (say, finding the root cause
of the issues, reporting, and fixing them), but this simple solution
works well for me. Also, this solution doesn't affect the build times
of the project (even when using ccache) since the targets built with
Gradle constitute a small part of the rest of the targets that are built
concurrently in a very robust manner.
Prior to this patch, Gradle would error out on follow-up invocations
after the build process has been interrupted, not being able to access
its build cache with an error like below:
--------------------------------------------------
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':kudu-hive:compileJava'.
> Could not create service of type DefaultGeneralCompileCaches using GradleScopeCompileServices.createGeneralCompileCaches().
> Cannot lock Java compile cache (...) as it has already been locked by this process.
--------------------------------------------------
Prior to this patch, building the project would fail sometimes when
invoking 'cmake -jN' (where N > 1) with errors like below while Gradle
was generating kudu-subprocess.jar and hms-plugin.jar concurrently:
--------------------------------------------------
...
[ 4%] Generating ../../../bin/kudu-subprocess.jar
...
[ 6%] Generating ../../../bin/hms-plugin.jar
...
java.lang.ArrayIndexOutOfBoundsException: 6
at shadow.org.objectweb.asm.ClassReader.readShort(ClassReader.java:3573)
...
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(Thread.java:750)
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':kudu-client:shadowJar'.
> 6
...
BUILD FAILED in 1m 0s
make[2]: *** [bin/hms-plugin.jar] Error 1
make[1]: *** [src/kudu/hms/CMakeFiles/hms_plugin_jar.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
java.lang.ArrayIndexOutOfBoundsException: 6
at shadow.org.objectweb.asm.ClassReader.readShort(ClassReader.java:3573)
...
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnabl
e.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(Thread.java:750)
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':kudu-subprocess:shadowJar'.
> 6
...
BUILD FAILED in 1m 3s
make[2]: *** [bin/kudu-subprocess.jar] Error 1
make[1]: *** [src/kudu/subprocess/CMakeFiles/subprocess_jar.dir/all] Error 2
make: *** [all] Error 2
--------------------------------------------------
Change-Id: Ib7d1637854233592ede5388bfc2287df96ec9865
Reviewed-on: http://gerrit.cloudera.org:8080/21626
Reviewed-by: Abhishek Chennaka <ac...@cloudera.com>
Tested-by: Alexey Serbin <al...@apache.org>
---
CMakeLists.txt | 20 ++++++++++++++++++--
1 file changed, 18 insertions(+), 2 deletions(-)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 0f6201447..3ebdd1b1a 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -109,8 +109,24 @@ set(JAVA_DIR ${CMAKE_CURRENT_SOURCE_DIR}/java)
# This solves the problem of escaped spaces in the EXTRA_GRADLE_FLAGS environment variable.
list(APPEND GRADLE_FLAGS $ENV{EXTRA_GRADLE_FLAGS})
separate_arguments(GRADLE_FLAGS)
-# We always want Gradle to use the plain console and quiet flag when called from cmake.
-list(APPEND GRADLE_FLAGS --quiet --console=plain)
+# When invoking Gradle:
+# * Log errors only, reducing the verbosity of the Gradle's output
+# * Make the output better fit for plain consoles, avoiding garbled logs
+# * Avoid building projects in parallel: when the kudu-subprocess.jar and
+# the hms-plugin.jar targets are built in parallel, it often ends up with
+# java.lang.ArrayIndexOutOfBoundsException while building shadowJar
+# for corresponding Java dependencies, and while it's not yet clear whether
+# the issue is in shadowJar implementation or elsewhere, it's quite simple
+# to address the issue by not building multiple Gradle targets concurrently
+# * Disable the build cache: sometimes, once a build process is interrupted,
+# follow-up Gradle invocations end up with an error like below, and while
+# it's not yet clear whether the issue is in not properly invoking Gradle
+# or it's a bug in signal handling elsewhere, it's quite simple to address
+# the issue by not using the Gradle's build cache
+#
+# Cannot lock Java compile cache (...) as it has already been locked by this process.
+#
+list(APPEND GRADLE_FLAGS --quiet --console=plain --no-parallel --no-build-cache)
list(REMOVE_DUPLICATES GRADLE_FLAGS)
# Initialize Gradle to ensure the wrapper is downloaded.