You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datasketches.apache.org by dc...@apache.org on 2021/09/01 15:48:09 UTC
[datasketches-memory] branch single-build-script updated (0af5fdd
-> e21e0a5)
This is an automated email from the ASF dual-hosted git repository.
dcromberge pushed a change to branch single-build-script
in repository https://gitbox.apache.org/repos/asf/datasketches-memory.git.
discard 0af5fdd Add documentation for standalone JAR script
discard 84158a7 Add checks to verify script packaged JAR
discard 89131e2 Minor cleanup
discard ac015ee Add git properties to JAR Manifest.MF
discard 959e104 Add attribution for version parsing
discard 6c9e666 Rename JavaHome to JDKHome
discard fb52cf6 Rename Version to GitTag
discard 234ff50 Move scripts to tools directory
discard 323275e Fix comment wrt optional input params
discard 3a1d7b8 WIP - Add build script to compile JAR for single JVM
add 5e31179 Changes to fix circular jar dependencies.
add f1c4b08 Restore memory dependency in resources module
new e2a91f1 WIP - Add build script to compile JAR for single JVM
new af890b3 Fix comment wrt optional input params
new eba1f74 Move scripts to tools directory
new 7628806 Rename Version to GitTag
new cd8993d Rename JavaHome to JDKHome
new 8d5f520 Add attribution for version parsing
new 36a8bc5 Add git properties to JAR Manifest.MF
new dd3299b Minor cleanup
new cbaff45 Add checks to verify script packaged JAR
new e21e0a5 Add documentation for standalone JAR script
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (0af5fdd)
\
N -- N -- N refs/heads/single-build-script (e21e0a5)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
datasketches-memory-java8-tests/pom.xml | 2 +-
.../apache/datasketches/memory/test/AllocateDirectMapMemoryTest.java | 2 +-
datasketches-memory-java9/pom.xml | 1 +
datasketches-memory-resources/pom.xml | 3 +++
pom.xml | 1 +
5 files changed, 7 insertions(+), 2 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-memory] 09/10: Add checks to verify script packaged
JAR
Posted by dc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dcromberge pushed a commit to branch single-build-script
in repository https://gitbox.apache.org/repos/asf/datasketches-memory.git
commit cbaff4525cf0d2dd0cb5dc3cfd3d3959d3fde768
Author: David Cromberge <dc...@apache.org>
AuthorDate: Wed Sep 1 15:37:33 2021 +0100
Add checks to verify script packaged JAR
---
.../memory/internal/NioBitsFields.java | 2 +-
tools/scripts/CheckMemoryJar.java | 99 ++++++++++++++++++++++
tools/scripts/compile-package-jar.sh | 46 +++++++---
3 files changed, 136 insertions(+), 11 deletions(-)
diff --git a/datasketches-memory-java9/src/main/java/org/apache/datasketches/memory/internal/NioBitsFields.java b/datasketches-memory-java9/src/main/java/org/apache/datasketches/memory/internal/NioBitsFields.java
index a2771bf..56f1e20 100644
--- a/datasketches-memory-java9/src/main/java/org/apache/datasketches/memory/internal/NioBitsFields.java
+++ b/datasketches-memory-java9/src/main/java/org/apache/datasketches/memory/internal/NioBitsFields.java
@@ -24,7 +24,7 @@ package org.apache.datasketches.memory.internal;
* Some field names in the VM internal class have changed in
* later versions. The appropriate class will be loaded by the class loader
* depending on the Java version that is used.
- * * For more information, see: https://openjdk.java.net/jeps/238
+ * For more information, see: https://openjdk.java.net/jeps/238
*/
class NioBitsFields {
static String COUNT_FIELD_NAME = "count";
diff --git a/tools/scripts/CheckMemoryJar.java b/tools/scripts/CheckMemoryJar.java
new file mode 100644
index 0000000..8504286
--- /dev/null
+++ b/tools/scripts/CheckMemoryJar.java
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.datasketches.memory.tools.scripts;
+
+import java.io.File;
+
+import org.apache.datasketches.memory.MapHandle;
+import org.apache.datasketches.memory.Memory;
+import org.apache.datasketches.memory.WritableHandle;
+import org.apache.datasketches.memory.WritableMemory;
+
+public class CheckMemoryJar {
+
+ public void checkHeapWritableMemory() {
+ try {
+ String str = "1 - Heap WritableMemory Successful";
+ WritableMemory mem = WritableMemory.allocate(2 * str.length());
+ writeReadAndPrintString(mem, str);
+ } catch (Exception ex) {
+ exitOnError("Heap Writable Memory", ex);
+ }
+ }
+
+ public void checkAllocateDirect() throws Exception {
+ try {
+ String str = "2 - Allocate Direct Successful";
+ WritableHandle wh = WritableMemory.allocateDirect(2 * str.length());
+ WritableMemory wmem = wh.getWritable();
+ writeReadAndPrintString(wmem, str);
+ wh.close();
+ } catch (Exception ex) {
+ exitOnError("Allocate Direct", ex);
+ }
+ }
+
+ public void checkMap(String mappedFilePath) throws Exception {
+ try {
+ String str = "3 - Memory Map Successful";
+ File file = new File(mappedFilePath);
+ MapHandle mh = Memory.map(file);
+ Memory mem = mh.get();
+ mh.close();
+ println(str);
+ } catch (Exception ex) {
+ exitOnError("Memory Map", ex);
+ }
+ }
+
+ /**********************/
+
+ private static void writeReadAndPrintString(WritableMemory wmem, String str) {
+ int len = str.length();
+ char[] cArr1 = str.toCharArray();
+ wmem.putCharArray(0, cArr1, 0, len);
+ char[] cArr2 = new char[len];
+ wmem.getCharArray(0, cArr2, 0, len);
+ String s2 = String.valueOf(cArr2);
+ println(s2);
+ }
+
+ private static void exitOnError(String checkName, Exception ex){
+ println(checkName + " check failed. Error: " + ex.toString());
+ System.exit(1);
+ }
+
+ public static void main(final String[] args) throws Exception {
+ if (args.length < 1) {
+ System.out.println("Please provide the full path to the memory mapped file!");
+ System.exit(1);
+ }
+
+ String mappedFilePath = args[0];
+ CheckMemoryJar check = new CheckMemoryJar();
+ check.checkHeapWritableMemory();
+ check.checkAllocateDirect();
+ check.checkMap(mappedFilePath);
+ println("");
+ println("All checks passed.");
+ }
+
+ static void println(Object obj) { System.out.println(obj.toString()); }
+}
diff --git a/tools/scripts/compile-package-jar.sh b/tools/scripts/compile-package-jar.sh
index 42163fb..a63d4be 100755
--- a/tools/scripts/compile-package-jar.sh
+++ b/tools/scripts/compile-package-jar.sh
@@ -38,14 +38,14 @@ if [ -z "$1" ]; then echo "Missing JDK home"; exit 1; fi
if [ -z "$2" ]; then echo "Missing Git Tag"; exit 1; fi
if [ -z "$3" ]; then echo "Missing project.basedir"; exit 1; fi
-#### Extract JDKHome and Version from input parameters ####
+#### Extract JDKHome, Version and ProjectBaseDir from input parameters ####
JDKHome=$1
GitTag=$2
+ProjectBaseDir=$3 #this must be an absolute path
#### Setup absolute directory references ####
-ProjectBaseDir=$3 #this must be an absolute path
-ScriptsDir=${ProjectBaseDir}/tools/scripts/
ProjectArtifactId="memory"
+ScriptsDir=${ProjectBaseDir}/tools/scripts/
#### Initialise path dependent variables ####
OutputDir=target
@@ -53,6 +53,7 @@ OutputJar=${OutputDir}/org.apache.datasketches.memory-${GitTag}.jar
PackageDir=${OutputDir}/package
PackageSrc=${PackageDir}/src
+PackageTests=${PackageDir}/test-classes
PackageContents=${PackageDir}/contents
PackageMeta=${PackageContents}/META-INF
PackageManifest=${PackageMeta}/MANIFEST.MF
@@ -97,8 +98,10 @@ fi
#### Cleanup and setup output directories ####
echo
echo "--- CLEAN & COMPILE ---"
-rm -r $OutputDir
+if [ -d "$OutputDir" ]; then rm -r $OutputDir; fi
+mkdir -p $OutputDir
mkdir -p $PackageSrc
+mkdir -p $PackageTests
mkdir -p $PackageContents
mkdir -p $PackageMeta
@@ -117,7 +120,7 @@ EOF
echo "$($ScriptsDir/getGitProperties.sh $ProjectBaseDir $ProjectArtifactId $GitTag)" >> $PackageManifest
#### Copy base tree to target/src
-rsync -a $MemoryJava8Src $PackageSrc
+rsync -a -I $MemoryJava8Src $PackageSrc
# version too low
if [[ $JavaVersion -lt 8 ]]; then
@@ -132,7 +135,7 @@ elif [[ $JavaVersion -lt 9 ]]; then
elif [[ $JavaVersion -lt 11 ]]; then
echo "Compiling with JDK version $JavaVersion..."
#### Copy java 9 src tree to target/src, overwriting replacements
- rsync -a $MemoryJava9Src $PackageSrc
+ rsync -a -I $MemoryJava9Src $PackageSrc
# Compile with JPMS exports
${Javac_} \
--add-exports java.base/jdk.internal.ref=org.apache.datasketches.memory \
@@ -143,9 +146,9 @@ elif [[ $JavaVersion -lt 11 ]]; then
elif [[ $JavaVersion -lt 14 ]]; then
echo "Compiling with JDK version $JavaVersion..."
#### Copy java 9 src tree to target/src, overwriting replacements
- rsync -a $MemoryJava9Src $PackageSrc
+ rsync -a -I $MemoryJava9Src $PackageSrc
#### Copy java 11 src tree to target/src, overwriting replacements
- rsync -a $MemoryJava11Src $PackageSrc
+ rsync -a -I $MemoryJava11Src $PackageSrc
# Compile with JPMS exports
${Javac_} \
--add-exports java.base/jdk.internal.ref=org.apache.datasketches.memory \
@@ -161,7 +164,30 @@ echo "--- JAR ---"
echo "Building JAR from ${PackageContents}..."
${Jar_} cf $OutputJar -C $PackageContents .
echo
-echo "--- JAR CONTENTS ---"
-${Jar_} tf ${OutputJar}
+# Uncomment for debugging purposes
+# echo "--- JAR CONTENTS ---"
+# ${Jar_} tf ${OutputJar}
+# echo
+echo "--- RUN JAR CHECKS ---"
+echo
+MemoryMapFile=$ScriptsDir/CheckMemoryJar.java # Memory map the textual source code for the JAR checks
+
+if [[ $JavaVersion -eq 8 ]]; then
+ ${Javac_} -cp $OutputJar -d $PackageTests $(find $ScriptsDir -name '*.java')
+
+ ${Java_} -cp $PackageTests:$OutputJar org.apache.datasketches.memory.tools.scripts.CheckMemoryJar $MemoryMapFile
+else
+ ${Javac_} \
+ --add-modules org.apache.datasketches.memory \
+ -p "$OutputJar" -d $PackageTests $(find $ScriptsDir -name '*.java')
+
+ ${Java_} \
+ --add-modules org.apache.datasketches.memory \
+ --add-exports java.base/jdk.internal.misc=org.apache.datasketches.memory \
+ --add-exports java.base/jdk.internal.ref=org.apache.datasketches.memory \
+ --add-opens java.base/java.nio=org.apache.datasketches.memory \
+ --add-opens java.base/sun.nio.ch=org.apache.datasketches.memory \
+ -p $OutputJar -cp $PackageTests org.apache.datasketches.memory.tools.scripts.CheckMemoryJar $MemoryMapFile
+fi
echo
echo "Successfully built ${OutputJar}"
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-memory] 05/10: Rename JavaHome to JDKHome
Posted by dc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dcromberge pushed a commit to branch single-build-script
in repository https://gitbox.apache.org/repos/asf/datasketches-memory.git
commit cd8993d525cd9edb9e01fa157455ed1b4ce92056
Author: David Cromberge <dc...@apache.org>
AuthorDate: Thu Aug 26 12:00:04 2021 +0100
Rename JavaHome to JDKHome
---
tools/scripts/compile-package-jar.sh | 33 ++++++++++++++++-----------------
1 file changed, 16 insertions(+), 17 deletions(-)
diff --git a/tools/scripts/compile-package-jar.sh b/tools/scripts/compile-package-jar.sh
index 9336f9c..fff53cd 100755
--- a/tools/scripts/compile-package-jar.sh
+++ b/tools/scripts/compile-package-jar.sh
@@ -17,16 +17,16 @@
# specific language governing permissions and limitations
# under the License.
-# This is a general bash script to build a java version-specific
+# This is a general bash script to build a JDK version-specific
# datasketches-memory-X.jar without multi-release functionality.
# This is intended to be used for developers compiling from source
-# who do not wish to install several versions of java on their
+# who do not wish to install several versions of the JDK on their
# machine.
# The script does not assume a POM file and does not use Maven.
# It does use git, SVN, and uses the script getGitProperties.sh
# Required Input Parameters:
-# \$1 = absolute path of Java home directory
+# \$1 = absolute path of JDK home directory
# \$2 = Git Version Tag for this deployment
# Example tag for SNAPSHOT : 1.0.0-SNAPSHOT
# Example tag for Release Candidate: 1.0.0-RC1
@@ -34,12 +34,12 @@
# \$3 = absolute path of project.basedir
# For example: $ <this script>.sh $JAVA_HOME 2.0.0-RC1 .
-if [ -z "$1" ]; then echo "Missing java.home"; exit 1; fi
+if [ -z "$1" ]; then echo "Missing JDK home"; exit 1; fi
if [ -z "$2" ]; then echo "Missing Git Tag"; exit 1; fi
if [ -z "$3" ]; then echo "Missing project.basedir"; exit 1; fi
-## Extract JavaHome and Version from input parameters:
-JavaHome=$1
+## Extract JDKHome and Version from input parameters:
+JDKHome=$1
GitTag=$2
# Setup absolute directory references
@@ -50,20 +50,20 @@ ProjectBaseDir=$3 #this must be an absolute path
cd ${ProjectBaseDir}
#### Use JAVA_HOME to set required executables ####
-if [[ -n "$JavaHome" ]] && [[ -x "${JavaHome}/bin/java" ]]; then
- Java_="${JavaHome}/bin/java"
+if [[ -n "$JDKHome" ]] && [[ -x "${JDKHome}/bin/java" ]]; then
+ Java_="${JDKHome}/bin/java"
else
echo "No java version could be found."; exit 1;
fi
-if [[ -n "$JavaHome" ]] && [[ -x "${JavaHome}/bin/javac" ]]; then
- Javac_="${JavaHome}/bin/javac"
+if [[ -n "$JDKHome" ]] && [[ -x "${JDKHome}/bin/javac" ]]; then
+ Javac_="${JDKHome}/bin/javac"
else
echo "No javac version could be found."; exit 1;
fi
-if [[ -n "$JavaHome" ]] && [[ -x "${JavaHome}/bin/jar" ]]; then
- Jar_="${JavaHome}/bin/jar"
+if [[ -n "$JDKHome" ]] && [[ -x "${JDKHome}/bin/jar" ]]; then
+ Jar_="${JDKHome}/bin/jar"
else
echo "No jar version could be found."; exit 1;
fi
@@ -72,7 +72,7 @@ fi
if [[ "$Java_" ]]; then
JavaVersion=$("$Java_" -version 2>&1 | head -1 | cut -d'"' -f2 | sed '/^1\./s///' | cut -d'.' -f1)
else
- echo "No version information could be determined from installed java."; exit 1;
+ echo "No version information could be determined from installed JDK."; exit 1;
fi
#### Cleanup and setup output directories ####
@@ -108,12 +108,12 @@ if [[ $JavaVersion -lt 8 ]]; then
# version 8
elif [[ $JavaVersion -lt 9 ]]; then
- echo "Compiling with java version $JavaVersion..."
+ echo "Compiling with JDK version $JavaVersion..."
${Javac_} -d $PackageContents $(find $PackageSrc -name '*.java')
# version 9 or 10
elif [[ $JavaVersion -lt 11 ]]; then
- echo "Compiling with java version $JavaVersion..."
+ echo "Compiling with JDK version $JavaVersion..."
#### Copy java 9 src tree to target/src, overwriting replacements
rsync -a $MemoryJava9Src $PackageSrc
# Compile with JPMS exports
@@ -124,7 +124,7 @@ elif [[ $JavaVersion -lt 11 ]]; then
# version 11, 12 or 13
elif [[ $JavaVersion -lt 14 ]]; then
- echo "Compiling with java version $JavaVersion..."
+ echo "Compiling with JDK version $JavaVersion..."
#### Copy java 9 src tree to target/src, overwriting replacements
rsync -a $MemoryJava9Src $PackageSrc
#### Copy java 11 src tree to target/src, overwriting replacements
@@ -148,4 +148,3 @@ echo "--- JAR CONTENTS ---"
${Jar_} tf ${OutputJar}
echo
echo "Successfully built ${OutputJar}"
-
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-memory] 10/10: Add documentation for standalone JAR
script
Posted by dc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dcromberge pushed a commit to branch single-build-script
in repository https://gitbox.apache.org/repos/asf/datasketches-memory.git
commit e21e0a594fa27e6c6c796e7daba70d4e183ad736
Author: David Cromberge <dc...@apache.org>
AuthorDate: Wed Sep 1 16:46:03 2021 +0100
Add documentation for standalone JAR script
---
README.md | 22 +++++++++++++++++++++-
1 file changed, 21 insertions(+), 1 deletion(-)
diff --git a/README.md b/README.md
index f57d32a..18d1ae2 100644
--- a/README.md
+++ b/README.md
@@ -65,12 +65,18 @@ __NOTE:__
For more information, see this [Maven Reactor Issue](https://issues.apache.org/jira/browse/MNG-3283).
### JDK versions required to compile
-This DataSketches component is pure Java and requires the following JDKs to compile:
+This DataSketches component is pure Java and requires any a JDK hotspot version between 8 and 13 to compile.
+Refer to the section below for instructions on how to build a standalone JAR using a single JDK version.
+
+### JDK versions required to compile the Multi-release JAR
+The following versions of the JDK are all required to compile the multi-release JAR:
- JDK8/Hotspot
- JDK9/Hotspot
- JDK11/Hotspot
+These versions are also required in order to build the Maven project in an IDE.
+
Ensure that your local environment has been configured according to the
[Maven Toolchains Configuration](docs/maven-toolchains.md).
@@ -112,6 +118,20 @@ This will create the following Jars:
* datasketches-memory-X.Y.Z-test-sources.jar The test source files
* datasketches-memory-X.Y.Z-javadoc.jar The compressed Javadocs.
+### Building a standalone JAR using a single version of the JDK
+A shell script is provided to compile and build a JAR for situations where it is not possible to
+install multiple versions of the JDK in the same build environment.
+
+This script can be found in the **tools/scripts** directory and is invoked from the command line as follows:
+
+ $ ./tools/scripts/compile-package-jar.sh JAVA_HOME GIT_VERSION PROJECT_BASE_DIR
+
+Where:
+
+1) The first argument is the absolute path of JDK home directory e.g. $JAVA_HOME
+2) The second argument is the Git Version Tag for this deployment e.g. 1.0.0-SNAPSHOT, 1.0.0-RC1, 1.0.0 etc.
+3) The third argument is the absolute path of project.basedir e.g. /src/apache-datasketches-memory
+
### Toolchains
This project makes use of Maven toolchains to ensure that the correct Java compiler version is
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-memory] 08/10: Minor cleanup
Posted by dc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dcromberge pushed a commit to branch single-build-script
in repository https://gitbox.apache.org/repos/asf/datasketches-memory.git
commit dd3299bfdc07737d50256c7ab67b100dd48ab31e
Author: David Cromberge <dc...@apache.org>
AuthorDate: Thu Aug 26 15:25:34 2021 +0100
Minor cleanup
---
tools/scripts/compile-package-jar.sh | 35 ++++++++++++++++++-----------------
1 file changed, 18 insertions(+), 17 deletions(-)
diff --git a/tools/scripts/compile-package-jar.sh b/tools/scripts/compile-package-jar.sh
index 5cb4940..42163fb 100755
--- a/tools/scripts/compile-package-jar.sh
+++ b/tools/scripts/compile-package-jar.sh
@@ -38,16 +38,30 @@ if [ -z "$1" ]; then echo "Missing JDK home"; exit 1; fi
if [ -z "$2" ]; then echo "Missing Git Tag"; exit 1; fi
if [ -z "$3" ]; then echo "Missing project.basedir"; exit 1; fi
-## Extract JDKHome and Version from input parameters:
+#### Extract JDKHome and Version from input parameters ####
JDKHome=$1
GitTag=$2
-# Setup absolute directory references
+#### Setup absolute directory references ####
ProjectBaseDir=$3 #this must be an absolute path
ScriptsDir=${ProjectBaseDir}/tools/scripts/
ProjectArtifactId="memory"
-####Move to project directory####
+#### Initialise path dependent variables ####
+OutputDir=target
+OutputJar=${OutputDir}/org.apache.datasketches.memory-${GitTag}.jar
+
+PackageDir=${OutputDir}/package
+PackageSrc=${PackageDir}/src
+PackageContents=${PackageDir}/contents
+PackageMeta=${PackageContents}/META-INF
+PackageManifest=${PackageMeta}/MANIFEST.MF
+
+MemoryJava8Src=datasketches-memory-java8/src/main/java
+MemoryJava9Src=datasketches-memory-java9/src/main/java
+MemoryJava11Src=datasketches-memory-java11/src/main/java
+
+#### Move to project directory ####
cd ${ProjectBaseDir}
#### Use JAVA_HOME to set required executables ####
@@ -81,19 +95,6 @@ else
fi
#### Cleanup and setup output directories ####
-OutputDir=target
-OutputJar=${OutputDir}/org.apache.datasketches.memory-${GitTag}.jar
-
-PackageDir=${OutputDir}/package
-PackageSrc=${PackageDir}/src
-PackageContents=${PackageDir}/contents
-PackageMeta=${PackageContents}/META-INF
-PackageManifest=${PackageMeta}/MANIFEST.MF
-
-MemoryJava8Src=datasketches-memory-java8/src/main/java
-MemoryJava9Src=datasketches-memory-java9/src/main/java
-MemoryJava11Src=datasketches-memory-java11/src/main/java
-
echo
echo "--- CLEAN & COMPILE ---"
rm -r $OutputDir
@@ -157,7 +158,7 @@ else
fi
echo
echo "--- JAR ---"
-echo "Building Jar from ${PackageContents}..."
+echo "Building JAR from ${PackageContents}..."
${Jar_} cf $OutputJar -C $PackageContents .
echo
echo "--- JAR CONTENTS ---"
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-memory] 01/10: WIP - Add build script to compile JAR
for single JVM
Posted by dc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dcromberge pushed a commit to branch single-build-script
in repository https://gitbox.apache.org/repos/asf/datasketches-memory.git
commit e2a91f136701827817f1757243d68fc5b28bd1d1
Author: David Cromberge <dc...@apache.org>
AuthorDate: Wed Aug 25 19:42:17 2021 +0100
WIP - Add build script to compile JAR for single JVM
---
scripts/compile-package-jar.sh | 151 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 151 insertions(+)
diff --git a/scripts/compile-package-jar.sh b/scripts/compile-package-jar.sh
new file mode 100755
index 0000000..5e8ad35
--- /dev/null
+++ b/scripts/compile-package-jar.sh
@@ -0,0 +1,151 @@
+#!/bin/bash -e
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This is a general bash script to build a java version-specific
+# datasketches-memory-X.jar without multi-release functionality.
+# This is intended to be used for developers compiling from source
+# who do not wish to install several versions of java on their
+# machine.
+# The script does not assume a POM file and does not use Maven.
+# It does use git, SVN, and uses the script getGitProperties.sh
+
+# Optional Input Parameters:
+# \$1 = absolute path of Java home directory
+# \$2 = Git Version Tag for this deployment
+# Example tag for SNAPSHOT : 1.0.0-SNAPSHOT
+# Example tag for Release Candidate: 1.0.0-RC1
+# Example tag for Release : 1.0.0
+# \$3 = absolute path of project.basedir
+# For example: $ <this script>.sh $JAVA_HOME 2.0.0-RC1 .
+
+if [ -z "$1" ]; then echo "Missing java.home"; exit 1; fi
+if [ -z "$2" ]; then echo "Missing Git Version"; exit 1; fi
+if [ -z "$3" ]; then echo "Missing project.basedir"; exit 1; fi
+
+# Setup absolute directory references
+ScriptsDir=$(pwd)
+
+## Extract JavaHome and Version from input parameters:
+JavaHome=$1
+Version=$2
+ProjectBaseDir=$3 #this must be an absolute path
+
+####Move to project directory####
+cd ${ProjectBaseDir}
+
+#### Use JAVA_HOME to set required executables ####
+if [[ -n "$JavaHome" ]] && [[ -x "${JavaHome}/bin/java" ]]; then
+ Java_="${JavaHome}/bin/java"
+else
+ echo "No java version could be found."; exit 1;
+fi
+
+if [[ -n "$JavaHome" ]] && [[ -x "${JavaHome}/bin/javac" ]]; then
+ Javac_="${JavaHome}/bin/javac"
+else
+ echo "No javac version could be found."; exit 1;
+fi
+
+if [[ -n "$JavaHome" ]] && [[ -x "${JavaHome}/bin/jar" ]]; then
+ Jar_="${JavaHome}/bin/jar"
+else
+ echo "No jar version could be found."; exit 1;
+fi
+
+#### Parse java -version into major version number ####
+if [[ "$Java_" ]]; then
+ JavaVersion=$("$Java_" -version 2>&1 | head -1 | cut -d'"' -f2 | sed '/^1\./s///' | cut -d'.' -f1)
+else
+ echo "No version information could be determined from installed java."; exit 1;
+fi
+
+#### Cleanup and setup output directories ####
+OutputDir=target
+OutputJar=${OutputDir}/org.apache.datasketches.memory-${Version}.jar
+
+PackageDir=${OutputDir}/package
+PackageSrc=${PackageDir}/src
+PackageContents=${PackageDir}/contents
+PackageMeta=${PackageDir}/META-INF
+
+MemoryJava8Src=datasketches-memory-java8/src/main/java
+MemoryJava9Src=datasketches-memory-java9/src/main/java
+MemoryJava11Src=datasketches-memory-java11/src/main/java
+
+echo
+echo "--- CLEAN & COMPILE ---"
+rm -r $OutputDir
+mkdir -p $PackageSrc
+mkdir -p $PackageContents
+mkdir -p $PackageMeta
+
+#### Copy LICENSE and NOTICE ####
+cp LICENSE $PackageMeta
+cp NOTICE $PackageMeta
+
+#### Copy base tree to target/src
+rsync -a $MemoryJava8Src $PackageSrc
+
+# version too low
+if [[ $JavaVersion -lt 8 ]]; then
+ echo "Java version not supported: " $JavaVersion; exit 1;
+
+# version 8
+elif [[ $JavaVersion -lt 9 ]]; then
+ echo "Compiling with java version $JavaVersion..."
+ ${Javac_} -d $PackageContents $(find $PackageSrc -name '*.java')
+
+# version 9 or 10
+elif [[ $JavaVersion -lt 11 ]]; then
+ echo "Compiling with java version $JavaVersion..."
+ #### Copy java 9 src tree to target/src, overwriting replacements
+ rsync -a $MemoryJava9Src $PackageSrc
+ # Compile with JPMS exports
+ ${Javac_} \
+ --add-exports java.base/jdk.internal.ref=org.apache.datasketches.memory \
+ --add-exports java.base/sun.nio.ch=org.apache.datasketches.memory \
+ -d $PackageContents $(find $PackageSrc -name '*.java')
+
+# version 11, 12 or 13
+elif [[ $JavaVersion -lt 14 ]]; then
+ echo "Compiling with java version $JavaVersion..."
+ #### Copy java 9 src tree to target/src, overwriting replacements
+ rsync -a $MemoryJava9Src $PackageSrc
+ #### Copy java 11 src tree to target/src, overwriting replacements
+ rsync -a $MemoryJava11Src $PackageSrc
+ # Compile with JPMS exports
+ ${Javac_} \
+ --add-exports java.base/jdk.internal.ref=org.apache.datasketches.memory \
+ --add-exports java.base/sun.nio.ch=org.apache.datasketches.memory \
+ -d $PackageContents $(find $PackageSrc -name '*.java')
+
+# version too high
+else
+ echo "Java version not supported: " $JavaVersion; exit 1;
+fi
+echo
+echo "--- JAR ---"
+echo "Building Jar from ${PackageContents}..."
+${Jar_} cf $OutputJar -C $PackageContents .
+echo
+echo "--- JAR CONTENTS ---"
+${Jar_} tf ${OutputJar}
+echo
+echo "Successfully built ${OutputJar}"
+
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-memory] 07/10: Add git properties to JAR Manifest.MF
Posted by dc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dcromberge pushed a commit to branch single-build-script
in repository https://gitbox.apache.org/repos/asf/datasketches-memory.git
commit 36a8bc547c9cad3375b467e52dc8771433d9be3b
Author: David Cromberge <dc...@apache.org>
AuthorDate: Thu Aug 26 15:20:55 2021 +0100
Add git properties to JAR Manifest.MF
---
tools/scripts/compile-package-jar.sh | 18 ++++++++--
tools/scripts/getGitProperties.sh | 70 ++++++++++++++++++++++++++++++++++++
2 files changed, 85 insertions(+), 3 deletions(-)
diff --git a/tools/scripts/compile-package-jar.sh b/tools/scripts/compile-package-jar.sh
index 5c1548f..5cb4940 100755
--- a/tools/scripts/compile-package-jar.sh
+++ b/tools/scripts/compile-package-jar.sh
@@ -23,7 +23,7 @@
# who do not wish to install several versions of the JDK on their
# machine.
# The script does not assume a POM file and does not use Maven.
-# It does use git, SVN, and uses the script getGitProperties.sh
+# It does use git and also uses the script getGitProperties.sh.
# Required Input Parameters:
# \$1 = absolute path of JDK home directory
@@ -43,8 +43,9 @@ JDKHome=$1
GitTag=$2
# Setup absolute directory references
-ScriptsDir=$(pwd)
ProjectBaseDir=$3 #this must be an absolute path
+ScriptsDir=${ProjectBaseDir}/tools/scripts/
+ProjectArtifactId="memory"
####Move to project directory####
cd ${ProjectBaseDir}
@@ -86,7 +87,8 @@ OutputJar=${OutputDir}/org.apache.datasketches.memory-${GitTag}.jar
PackageDir=${OutputDir}/package
PackageSrc=${PackageDir}/src
PackageContents=${PackageDir}/contents
-PackageMeta=${PackageDir}/META-INF
+PackageMeta=${PackageContents}/META-INF
+PackageManifest=${PackageMeta}/MANIFEST.MF
MemoryJava8Src=datasketches-memory-java8/src/main/java
MemoryJava9Src=datasketches-memory-java9/src/main/java
@@ -103,6 +105,16 @@ mkdir -p $PackageMeta
cp LICENSE $PackageMeta
cp NOTICE $PackageMeta
+#### Generate MANIFEST.MF ####
+cat >> ${PackageManifest}<< EOF
+Manifest-Version: 1.0
+Created-By: Apache Datasketches Memory compile-package-jar.sh
+Multi-Release: false
+EOF
+
+#### Generate git.properties file ####
+echo "$($ScriptsDir/getGitProperties.sh $ProjectBaseDir $ProjectArtifactId $GitTag)" >> $PackageManifest
+
#### Copy base tree to target/src
rsync -a $MemoryJava8Src $PackageSrc
diff --git a/tools/scripts/getGitProperties.sh b/tools/scripts/getGitProperties.sh
new file mode 100755
index 0000000..d75e0b3
--- /dev/null
+++ b/tools/scripts/getGitProperties.sh
@@ -0,0 +1,70 @@
+#!/bin/bash -e
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+MyBase=$(pwd)
+ProjectBaseDir=$1 #this must be an absolute path
+ArtId=$2
+Tag=$3
+
+####Move to project directory####
+cd ${ProjectBaseDir}
+
+CR=$'\n'
+
+#Add Implementation Vendor
+prop=$prop'Implementation-Vendor: '
+tmp='The Apache Software Foundataion'$CR
+prop=$prop$tmp
+
+#Add GroupId : ArtifactId
+prop=$prop'GroupId-ArtifactId: '
+tmp='org.apache.datasketches:'$ArtId$CR
+prop=$prop$tmp
+
+# Add Branch
+prop=$prop'git.branch: '
+tmp=''$(git rev-parse --abbrev-ref HEAD)''$CR
+prop=$prop$tmp
+
+#Add commit-id
+prop=$prop'git.commit.id.full: '
+ID=$(git rev-parse HEAD)
+tmp=''$ID''$CR
+prop=$prop$tmp
+
+#Add timestamp
+prop=$prop'git.commit.time: '
+tmp=''$(git show --no-patch --no-notes --pretty='%cI' $ID)''$CR
+prop=$prop$tmp
+
+#Add user email
+prop=$prop'git-commit-user-email: '
+tmp=''$(git show --no-patch --no-notes --pretty='%ce' $ID)''$CR
+prop=$prop$tmp
+
+#Add Tag
+prop=$prop'git.commit.tag: '
+tmp=''$Tag''$CR
+prop=$prop$tmp
+
+echo "$prop"
+
+
+
+
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-memory] 03/10: Move scripts to tools directory
Posted by dc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dcromberge pushed a commit to branch single-build-script
in repository https://gitbox.apache.org/repos/asf/datasketches-memory.git
commit eba1f746b1422cf27e7a0815afb67fccc5825a3f
Author: David Cromberge <dc...@apache.org>
AuthorDate: Thu Aug 26 11:38:39 2021 +0100
Move scripts to tools directory
---
{scripts => tools/scripts}/compile-package-jar.sh | 0
1 file changed, 0 insertions(+), 0 deletions(-)
diff --git a/scripts/compile-package-jar.sh b/tools/scripts/compile-package-jar.sh
similarity index 100%
rename from scripts/compile-package-jar.sh
rename to tools/scripts/compile-package-jar.sh
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-memory] 02/10: Fix comment wrt optional input params
Posted by dc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dcromberge pushed a commit to branch single-build-script
in repository https://gitbox.apache.org/repos/asf/datasketches-memory.git
commit af890b36a1a67fba4173d2d9121af20b37276fae
Author: David Cromberge <dc...@apache.org>
AuthorDate: Thu Aug 26 11:33:44 2021 +0100
Fix comment wrt optional input params
---
scripts/compile-package-jar.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/compile-package-jar.sh b/scripts/compile-package-jar.sh
index 5e8ad35..000f181 100755
--- a/scripts/compile-package-jar.sh
+++ b/scripts/compile-package-jar.sh
@@ -25,7 +25,7 @@
# The script does not assume a POM file and does not use Maven.
# It does use git, SVN, and uses the script getGitProperties.sh
-# Optional Input Parameters:
+# Required Input Parameters:
# \$1 = absolute path of Java home directory
# \$2 = Git Version Tag for this deployment
# Example tag for SNAPSHOT : 1.0.0-SNAPSHOT
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-memory] 06/10: Add attribution for version parsing
Posted by dc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dcromberge pushed a commit to branch single-build-script
in repository https://gitbox.apache.org/repos/asf/datasketches-memory.git
commit 8d5f520101e64a8af9bf49c67aa3f66c5ae2e984
Author: David Cromberge <dc...@apache.org>
AuthorDate: Thu Aug 26 12:29:08 2021 +0100
Add attribution for version parsing
---
tools/scripts/compile-package-jar.sh | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/tools/scripts/compile-package-jar.sh b/tools/scripts/compile-package-jar.sh
index fff53cd..5c1548f 100755
--- a/tools/scripts/compile-package-jar.sh
+++ b/tools/scripts/compile-package-jar.sh
@@ -70,6 +70,10 @@ fi
#### Parse java -version into major version number ####
if [[ "$Java_" ]]; then
+ # This expression extracts the correct major version of the Java runtime.
+ # For older runtime versions, such as 1.8, the leading '1.' is removed.
+ # Adapted from this answer on StackOverflow:
+ # https://stackoverflow.com/questions/7334754/correct-way-to-check-java-version-from-bash-script/56243046#56243046
JavaVersion=$("$Java_" -version 2>&1 | head -1 | cut -d'"' -f2 | sed '/^1\./s///' | cut -d'.' -f1)
else
echo "No version information could be determined from installed JDK."; exit 1;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-memory] 04/10: Rename Version to GitTag
Posted by dc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dcromberge pushed a commit to branch single-build-script
in repository https://gitbox.apache.org/repos/asf/datasketches-memory.git
commit 762880668e0f1e52f2e41294000b9059c701f79a
Author: David Cromberge <dc...@apache.org>
AuthorDate: Thu Aug 26 11:46:20 2021 +0100
Rename Version to GitTag
---
tools/scripts/compile-package-jar.sh | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/tools/scripts/compile-package-jar.sh b/tools/scripts/compile-package-jar.sh
index 000f181..9336f9c 100755
--- a/tools/scripts/compile-package-jar.sh
+++ b/tools/scripts/compile-package-jar.sh
@@ -34,16 +34,16 @@
# \$3 = absolute path of project.basedir
# For example: $ <this script>.sh $JAVA_HOME 2.0.0-RC1 .
-if [ -z "$1" ]; then echo "Missing java.home"; exit 1; fi
-if [ -z "$2" ]; then echo "Missing Git Version"; exit 1; fi
-if [ -z "$3" ]; then echo "Missing project.basedir"; exit 1; fi
-
-# Setup absolute directory references
-ScriptsDir=$(pwd)
+if [ -z "$1" ]; then echo "Missing java.home"; exit 1; fi
+if [ -z "$2" ]; then echo "Missing Git Tag"; exit 1; fi
+if [ -z "$3" ]; then echo "Missing project.basedir"; exit 1; fi
## Extract JavaHome and Version from input parameters:
JavaHome=$1
-Version=$2
+GitTag=$2
+
+# Setup absolute directory references
+ScriptsDir=$(pwd)
ProjectBaseDir=$3 #this must be an absolute path
####Move to project directory####
@@ -77,7 +77,7 @@ fi
#### Cleanup and setup output directories ####
OutputDir=target
-OutputJar=${OutputDir}/org.apache.datasketches.memory-${Version}.jar
+OutputJar=${OutputDir}/org.apache.datasketches.memory-${GitTag}.jar
PackageDir=${OutputDir}/package
PackageSrc=${PackageDir}/src
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org