You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by gr...@apache.org on 2019/01/30 20:45:52 UTC
[kudu] branch master updated (ebe6265 -> 9ac3b29)
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git.
from ebe6265 KUDU-2411: Add OS/Arch detection to binary extract
new f46d220 [build] Fix mini cluster binary relocation on Mac OS
new 4d4c5e1 [build] Improve mini cluster artifact name
new 9ac3b29 [gradle] Increase test JVMs heap size
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
build-support/build_mini_cluster_binaries.sh | 2 +-
build-support/relocate_binaries_for_mini_cluster.py | 17 +++++++++++++----
java/gradle/tests.gradle | 2 ++
3 files changed, 16 insertions(+), 5 deletions(-)
[kudu] 02/03: [build] Improve mini cluster artifact name
Posted by gr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 4d4c5e1f4b2348cd2552fe1d78372412bca134e7
Author: Grant Henke <gr...@apache.org>
AuthorDate: Wed Jan 30 13:13:56 2019 -0600
[build] Improve mini cluster artifact name
Adjust the name of the artifact/jar to include the OS and
architecture that matches the expected Maven format
for compatibility with the os-maven-plugin.
As an example, this artifact name matches the protoc
artifact name structure:
http://central.maven.org/maven2/com/google/protobuf/protoc/3.6.1/
Change-Id: I75007d3623aa3c3d06dea083d5d25f241ca1c73c
Reviewed-on: http://gerrit.cloudera.org:8080/12312
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Kudu Jenkins
---
build-support/build_mini_cluster_binaries.sh | 2 +-
build-support/relocate_binaries_for_mini_cluster.py | 11 +++++++++--
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/build-support/build_mini_cluster_binaries.sh b/build-support/build_mini_cluster_binaries.sh
index aac4e4a..32ec346 100755
--- a/build-support/build_mini_cluster_binaries.sh
+++ b/build-support/build_mini_cluster_binaries.sh
@@ -74,7 +74,7 @@ make -j$NUM_PROCS $TARGETS
# Relocate the binaries.
$SOURCE_ROOT/build-support/relocate_binaries_for_mini_cluster.py $BUILD_ROOT $TARGETS
-ARTIFACT_NAME=$(ls -d apache-kudu-* | sed 's#/##' | head -1)
+ARTIFACT_NAME=$(ls -d kudu-binary* | sed 's#/##' | head -1)
# Strip everything to minimize the size of the tarball we generate.
echo Stripping symbols...
diff --git a/build-support/relocate_binaries_for_mini_cluster.py b/build-support/relocate_binaries_for_mini_cluster.py
index aa12fc6..182ae8b 100755
--- a/build-support/relocate_binaries_for_mini_cluster.py
+++ b/build-support/relocate_binaries_for_mini_cluster.py
@@ -184,11 +184,18 @@ def get_dep_library_paths_macos(binary_path):
def get_artifact_name():
"""
- Read the Kudu version to create an archive with an appropriate name.
+ Create an archive with an appropriate name. Including version, OS, and architecture.
"""
+ if IS_LINUX:
+ os_str = "linux"
+ elif IS_MACOS:
+ os_str = "osx"
+ else:
+ raise NotImplementedError("Unsupported platform")
+ arch = os.uname()[4]
with open(os.path.join(SOURCE_ROOT, "version.txt"), 'r') as version:
version = version.readline().strip().decode("utf-8")
- artifact_name = "apache-kudu-%s" % (version, )
+ artifact_name = "kudu-binary-%s-%s-%s" % (version, os_str, arch)
return artifact_name
def mkconfig(build_root, artifact_root):
[kudu] 01/03: [build] Fix mini cluster binary relocation on Mac OS
Posted by gr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit f46d2202e19939d137be0e7ff1aad9525f2f54fa
Author: Grant Henke <gr...@apache.org>
AuthorDate: Wed Jan 30 10:04:46 2019 -0600
[build] Fix mini cluster binary relocation on Mac OS
On Mac OS the ssl libs like `libssl.1.0.0.dylib` are read only.
When relocate_binaries_for_mini_cluster.py tries to copy
these read only libs, the build fails because the file
already exists and cannot be replaced.
This patch changes the copy_file method to stop matching
the permissions of the copied files. Additionally it clears the
artifact root on every run to ensure old files are not left behind.
Change-Id: Ib653b175ba2d0e5563509aecac372c9829eca2f5
Reviewed-on: http://gerrit.cloudera.org:8080/12310
Reviewed-by: Mike Percy <mp...@apache.org>
Tested-by: Grant Henke <gr...@apache.org>
---
build-support/relocate_binaries_for_mini_cluster.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/build-support/relocate_binaries_for_mini_cluster.py b/build-support/relocate_binaries_for_mini_cluster.py
index 9b4cac8..aa12fc6 100755
--- a/build-support/relocate_binaries_for_mini_cluster.py
+++ b/build-support/relocate_binaries_for_mini_cluster.py
@@ -217,12 +217,11 @@ def prep_artifact_dirs(config):
def copy_file(src, dest):
"""
- Copy the file with path 'src' to path 'dest', including the file mode.
+ Copy the file with path 'src' to path 'dest'.
If 'src' is a symlink, the link will be followed and 'dest' will be written
as a plain file.
"""
shutil.copyfile(src, dest)
- shutil.copymode(src, dest)
def chrpath(target, new_rpath):
"""
@@ -344,6 +343,9 @@ def main():
artifact_name = get_artifact_name()
artifact_root = os.path.join(build_root, artifact_name)
+ # Clear the artifact root to ensure a clean build.
+ if os.path.exists(artifact_root):
+ shutil.rmtree(artifact_root)
logging.info("Including targets and their dependencies in archive...")
config = mkconfig(build_root, artifact_root)
[kudu] 03/03: [gradle] Increase test JVMs heap size
Posted by gr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 9ac3b295a9cf41316e100f5f1745d3c6120d6c48
Author: Hao Hao <ha...@cloudera.com>
AuthorDate: Thu Dec 27 16:41:00 2018 -0800
[gradle] Increase test JVMs heap size
The flume test RegexpKuduOperationsProducerParseErrorTest is flaky and
throws OutOfMemoryError. As the default heap size for test task is 512m
[1], this commit increases the maximum java heap size to 1g to avoid
such error.
[1]: https://docs.gradle.org/current/userguide/build_environment.html#sec:configuring_jvm_memory
Change-Id: Iae86fe493a1fdc337d5ca8615b043b437a2a116d
Reviewed-on: http://gerrit.cloudera.org:8080/12134
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Reviewed-by: Grant Henke <gr...@apache.org>
---
java/gradle/tests.gradle | 2 ++
1 file changed, 2 insertions(+)
diff --git a/java/gradle/tests.gradle b/java/gradle/tests.gradle
index 9c51858..2f43d96 100644
--- a/java/gradle/tests.gradle
+++ b/java/gradle/tests.gradle
@@ -38,6 +38,8 @@ tasks.withType(Test) {
// Enable assertions during tests.
jvmArgs += "-enableassertions"
+ // Config 1g maximum java heap size for test JVMs.
+ jvmArgs += "-Xmx1g"
// Hide the Java 9+ warnings about illegal reflective access
if (JavaVersion.current().isJava9Compatible()) {
jvmArgs += "--illegal-access=permit"