You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by ij...@apache.org on 2019/11/21 16:29:49 UTC
[kafka] branch 2.4 updated (cbc9f57 -> c19266a)
This is an automated email from the ASF dual-hosted git repository.
ijuma pushed a change to branch 2.4
in repository https://gitbox.apache.org/repos/asf/kafka.git.
from cbc9f57 HOTFIX: safely clear all active state in onPartitionsLost (#7691)
new d03f52c KAFKA-1714: Fix gradle wrapper bootstrapping (#6031)
new c19266a MINOR: Update gradle wrapper
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 3 +-
README.md | 10 +-
gradle/wrapper/gradle-wrapper.properties | 19 +++
gradlew | 215 +++++++++++++++++++++++++++++++
wrapper.gradle | 107 ++++++++++++++-
5 files changed, 339 insertions(+), 15 deletions(-)
create mode 100644 gradle/wrapper/gradle-wrapper.properties
create mode 100755 gradlew
[kafka] 02/02: MINOR: Update gradle wrapper
Posted by ij...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ijuma pushed a commit to branch 2.4
in repository https://gitbox.apache.org/repos/asf/kafka.git
commit c19266accbe46ba3fde9d40ee17313e631037463
Author: Ismael Juma <is...@juma.me.uk>
AuthorDate: Thu Nov 21 08:29:09 2019 -0800
MINOR: Update gradle wrapper
---
gradle/wrapper/gradle-wrapper.properties | 2 +-
gradlew | 24 ++++++++++++++++++++----
2 files changed, 21 insertions(+), 5 deletions(-)
diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties
index 3a8f801..a3f32a4 100644
--- a/gradle/wrapper/gradle-wrapper.properties
+++ b/gradle/wrapper/gradle-wrapper.properties
@@ -14,6 +14,6 @@
# limitations under the License.
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-5.0-all.zip
+distributionUrl=https\://services.gradle.org/distributions/gradle-5.6.2-all.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
diff --git a/gradlew b/gradlew
index f3276ba..db5089d 100755
--- a/gradlew
+++ b/gradlew
@@ -14,6 +14,22 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+#
+# Copyright 2015 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
##############################################################################
##
## Gradle start up script for UN*X
@@ -42,7 +58,7 @@ APP_NAME="Gradle"
APP_BASE_NAME=`basename "$0"`
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
-DEFAULT_JVM_OPTS='"-Xmx64m"'
+DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD="maximum"
@@ -82,7 +98,7 @@ esac
# Loop in case we encounter an error.
for attempt in 1 2 3; do
if [ ! -e $APP_HOME/gradle/wrapper/gradle-wrapper.jar ]; then
- if ! curl -s -S --retry 3 -L -o "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" "https://raw.githubusercontent.com/gradle/gradle/v5.0.0/gradle/wrapper/gradle-wrapper.jar"; then
+ if ! curl -s -S --retry 3 -L -o "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" "https://raw.githubusercontent.com/gradle/gradle/v5.6.2/gradle/wrapper/gradle-wrapper.jar"; then
rm -f "$APP_HOME/gradle/wrapper/gradle-wrapper.jar"
# Pause for a bit before looping in case the server throttled us.
sleep 5
@@ -136,8 +152,8 @@ if $darwin; then
GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
fi
-# For Cygwin, switch paths to Windows format before running java
-if $cygwin ; then
+# For Cygwin or MSYS, switch paths to Windows format before running java
+if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then
APP_HOME=`cygpath --path --mixed "$APP_HOME"`
CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
JAVACMD=`cygpath --unix "$JAVACMD"`
[kafka] 01/02: KAFKA-1714: Fix gradle wrapper bootstrapping (#6031)
Posted by ij...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ijuma pushed a commit to branch 2.4
in repository https://gitbox.apache.org/repos/asf/kafka.git
commit d03f52c91558d8bff44522ca5598d673d6c3935c
Author: Grant Henke <gr...@users.noreply.github.com>
AuthorDate: Thu Nov 21 10:06:06 2019 -0600
KAFKA-1714: Fix gradle wrapper bootstrapping (#6031)
Given we need to follow the Apache rule of not checking
any binaries into the source code, Kafka has always had
a bit of a tricky Gradle bootstrap.
Using ./gradlew as users expect doesn’t work and a
local and compatible version of Gradle was required to
generate the wrapper first.
This patch changes the behavior of the wrapper task to
instead generate a gradlew script that can bootstrap the
jar itself. Additionally it adds a license, removes the bat
script, and handles retries.
The documentation in the readme was also updated.
Going forward patches that upgrade gradle should run
`gradle wrapper` before checking in the change.
With this change users using ./gradlew can be sure they
are always building with the correct version of Gradle.
Reviewers: Viktor Somogyi <vi...@gmail.com>, Ismael Juma <ismael@juma.me.uk
---
.gitignore | 3 +-
README.md | 10 +-
gradle/wrapper/gradle-wrapper.properties | 19 +++
gradlew | 199 +++++++++++++++++++++++++++++++
wrapper.gradle | 107 ++++++++++++++++-
5 files changed, 323 insertions(+), 15 deletions(-)
diff --git a/.gitignore b/.gitignore
index 3daa2b8..f640e8d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -34,8 +34,7 @@ Vagrantfile.local
config/server-*
config/zookeeper-*
core/data/*
-gradle/wrapper/*
-gradlew
+gradle/wrapper/*.jar
gradlew.bat
results
diff --git a/README.md b/README.md
index 76fbc4e..b7081c4 100644
--- a/README.md
+++ b/README.md
@@ -2,20 +2,12 @@ Apache Kafka
=================
See our [web site](https://kafka.apache.org) for details on the project.
-You need to have [Gradle](https://www.gradle.org/installation) and [Java](https://www.oracle.com/technetwork/java/javase/downloads/index.html) installed.
-
-Kafka requires Gradle 5.0 or higher.
+You need to have [Java](http://www.oracle.com/technetwork/java/javase/downloads/index.html) installed.
Java 8 should be used for building in order to support both Java 8 and Java 11 at runtime.
Scala 2.12 is used by default, see below for how to use a different Scala version or all of the supported Scala versions.
-### First bootstrap and download the wrapper ###
- cd kafka_source_dir
- gradle
-
-Now everything else will work.
-
### Build a jar and run it ###
./gradlew jar
diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties
new file mode 100644
index 0000000..3a8f801
--- /dev/null
+++ b/gradle/wrapper/gradle-wrapper.properties
@@ -0,0 +1,19 @@
+#
+# Copyright 2017 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+distributionBase=GRADLE_USER_HOME
+distributionPath=wrapper/dists
+distributionUrl=https\://services.gradle.org/distributions/gradle-5.0-all.zip
+zipStoreBase=GRADLE_USER_HOME
+zipStorePath=wrapper/dists
diff --git a/gradlew b/gradlew
new file mode 100755
index 0000000..f3276ba
--- /dev/null
+++ b/gradlew
@@ -0,0 +1,199 @@
+#!/usr/bin/env sh
+#
+# Copyright 2017 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+##############################################################################
+##
+## Gradle start up script for UN*X
+##
+##############################################################################
+
+# Attempt to set APP_HOME
+# Resolve links: $0 may be a link
+PRG="$0"
+# Need this for relative symlinks.
+while [ -h "$PRG" ] ; do
+ ls=`ls -ld "$PRG"`
+ link=`expr "$ls" : '.*-> \(.*\)$'`
+ if expr "$link" : '/.*' > /dev/null; then
+ PRG="$link"
+ else
+ PRG=`dirname "$PRG"`"/$link"
+ fi
+done
+SAVED="`pwd`"
+cd "`dirname \"$PRG\"`/" >/dev/null
+APP_HOME="`pwd -P`"
+cd "$SAVED" >/dev/null
+
+APP_NAME="Gradle"
+APP_BASE_NAME=`basename "$0"`
+
+# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
+DEFAULT_JVM_OPTS='"-Xmx64m"'
+
+# Use the maximum available, or set MAX_FD != -1 to use that value.
+MAX_FD="maximum"
+
+warn () {
+ echo "$*"
+}
+
+die () {
+ echo
+ echo "$*"
+ echo
+ exit 1
+}
+
+# OS specific support (must be 'true' or 'false').
+cygwin=false
+msys=false
+darwin=false
+nonstop=false
+case "`uname`" in
+ CYGWIN* )
+ cygwin=true
+ ;;
+ Darwin* )
+ darwin=true
+ ;;
+ MINGW* )
+ msys=true
+ ;;
+ NONSTOP* )
+ nonstop=true
+ ;;
+esac
+
+
+# Loop in case we encounter an error.
+for attempt in 1 2 3; do
+ if [ ! -e $APP_HOME/gradle/wrapper/gradle-wrapper.jar ]; then
+ if ! curl -s -S --retry 3 -L -o "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" "https://raw.githubusercontent.com/gradle/gradle/v5.0.0/gradle/wrapper/gradle-wrapper.jar"; then
+ rm -f "$APP_HOME/gradle/wrapper/gradle-wrapper.jar"
+ # Pause for a bit before looping in case the server throttled us.
+ sleep 5
+ continue
+ fi
+ fi
+done
+
+CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
+
+# Determine the Java command to use to start the JVM.
+if [ -n "$JAVA_HOME" ] ; then
+ if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
+ # IBM's JDK on AIX uses strange locations for the executables
+ JAVACMD="$JAVA_HOME/jre/sh/java"
+ else
+ JAVACMD="$JAVA_HOME/bin/java"
+ fi
+ if [ ! -x "$JAVACMD" ] ; then
+ die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
+
+Please set the JAVA_HOME variable in your environment to match the
+location of your Java installation."
+ fi
+else
+ JAVACMD="java"
+ which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
+
+Please set the JAVA_HOME variable in your environment to match the
+location of your Java installation."
+fi
+
+# Increase the maximum file descriptors if we can.
+if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then
+ MAX_FD_LIMIT=`ulimit -H -n`
+ if [ $? -eq 0 ] ; then
+ if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
+ MAX_FD="$MAX_FD_LIMIT"
+ fi
+ ulimit -n $MAX_FD
+ if [ $? -ne 0 ] ; then
+ warn "Could not set maximum file descriptor limit: $MAX_FD"
+ fi
+ else
+ warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
+ fi
+fi
+
+# For Darwin, add options to specify how the application appears in the dock
+if $darwin; then
+ GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
+fi
+
+# For Cygwin, switch paths to Windows format before running java
+if $cygwin ; then
+ APP_HOME=`cygpath --path --mixed "$APP_HOME"`
+ CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
+ JAVACMD=`cygpath --unix "$JAVACMD"`
+
+ # We build the pattern for arguments to be converted via cygpath
+ ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
+ SEP=""
+ for dir in $ROOTDIRSRAW ; do
+ ROOTDIRS="$ROOTDIRS$SEP$dir"
+ SEP="|"
+ done
+ OURCYGPATTERN="(^($ROOTDIRS))"
+ # Add a user-defined pattern to the cygpath arguments
+ if [ "$GRADLE_CYGPATTERN" != "" ] ; then
+ OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
+ fi
+ # Now convert the arguments - kludge to limit ourselves to /bin/sh
+ i=0
+ for arg in "$@" ; do
+ CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
+ CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option
+
+ if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
+ eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
+ else
+ eval `echo args$i`="\"$arg\""
+ fi
+ i=$((i+1))
+ done
+ case $i in
+ (0) set -- ;;
+ (1) set -- "$args0" ;;
+ (2) set -- "$args0" "$args1" ;;
+ (3) set -- "$args0" "$args1" "$args2" ;;
+ (4) set -- "$args0" "$args1" "$args2" "$args3" ;;
+ (5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
+ (6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
+ (7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
+ (8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
+ (9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
+ esac
+fi
+
+# Escape application args
+save () {
+ for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
+ echo " "
+}
+APP_ARGS=$(save "$@")
+
+# Collect all arguments for the java command, following the shell quoting and substitution rules
+eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"
+
+# by default we should be in the correct project dir, but when run from Finder on Mac, the cwd is wrong
+if [ "$(uname)" = "Darwin" ] && [ "$HOME" = "$PWD" ]; then
+ cd "$(dirname "$0")"
+fi
+
+exec "$JAVACMD" "$@"
diff --git a/wrapper.gradle b/wrapper.gradle
index bc63506..3ce451a 100644
--- a/wrapper.gradle
+++ b/wrapper.gradle
@@ -17,9 +17,108 @@
* under the License.
*/
-defaultTasks 'downloadWrapper'
+// This file contains tasks for the gradle wrapper generation.
-task downloadWrapper(type: Wrapper) {
- description = "Download the gradle wrapper and requisite files. Overwrites existing wrapper files."
+// Ensure the wrapper script is generated based on the version defined in the project
+// and not the version installed on the machine running the task.
+// Read more about the wrapper here: https://docs.gradle.org/current/userguide/gradle_wrapper.html
+wrapper {
gradleVersion = project.gradleVersion
-}
\ No newline at end of file
+ distributionType = Wrapper.DistributionType.ALL
+}
+
+def licenseString = """#
+# Copyright 2017 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License."""
+
+// Custom task to inject support for downloading the gradle wrapper jar if it doesn't exist.
+// This allows us to avoid checking in the jar to our repository.
+// Additionally adds a license header to the wrapper while editing the file contents.
+task bootstrapWrapper() {
+ // In the doLast block so this runs when the task is called and not during project configuration.
+ doLast {
+ def wrapperBasePath = "\$APP_HOME/gradle/wrapper"
+ def wrapperJarPath = wrapperBasePath + "/gradle-wrapper.jar"
+
+ // Add a trailing zero to the version if needed.
+ def fullVersion = project.gradleVersion.count(".") == 1 ? "${project.gradleVersion}.0" : versions.gradle
+ // Leverages the wrapper jar checked into the gradle project on github because the jar isn't
+ // available elsewhere. Using raw.githubusercontent.com instead of github.com because
+ // github.com servers deprecated TLSv1/TLSv1.1 support some time ago, so older versions
+ // of curl (built against OpenSSL library that doesn't support TLSv1.2) would fail to
+ // fetch the jar.
+ def wrapperBaseUrl = "https://raw.githubusercontent.com/gradle/gradle/v$fullVersion/gradle/wrapper"
+ def wrapperJarUrl = wrapperBaseUrl + "/gradle-wrapper.jar"
+
+ def bootstrapString = """
+ # Loop in case we encounter an error.
+ for attempt in 1 2 3; do
+ if [ ! -e $wrapperJarPath ]; then
+ if ! curl -s -S --retry 3 -L -o "$wrapperJarPath" "$wrapperJarUrl"; then
+ rm -f "$wrapperJarPath"
+ # Pause for a bit before looping in case the server throttled us.
+ sleep 5
+ continue
+ fi
+ fi
+ done
+ """.stripIndent()
+
+ def wrapperScript = wrapper.scriptFile
+ def wrapperLines = wrapperScript.readLines()
+ wrapperScript.withPrintWriter { out ->
+ def licenseWritten = false
+ def bootstrapWritten = false
+ wrapperLines.each { line ->
+ // Print the wrapper bootstrap before the first usage of the wrapper jar.
+ if (!bootstrapWritten && line.contains("gradle-wrapper.jar")) {
+ out.println(bootstrapString)
+ bootstrapWritten = true
+ }
+ out.print(line)
+ // Print the licence after the shebang.
+ if(!licenseWritten && line.contains("#!/usr/bin/env sh")) {
+ out.println()
+ out.print(licenseString)
+ licenseWritten = true
+ }
+ out.println() // New Line
+ }
+ }
+ }
+}
+wrapper.finalizedBy bootstrapWrapper
+
+// Custom task to add a license header to the gradle-wrapper.properties file.
+task bootstrapWrapperProperties() {
+ // In the doLast block so this runs when the task is called and not during project configuration.
+ doLast {
+ def wrapperProperties = wrapper.propertiesFile
+ def wrapperLines = wrapperProperties.readLines()
+ wrapperProperties.withPrintWriter { out ->
+ // Print the license
+ out.println(licenseString)
+ wrapperLines.each { line ->
+ out.println(line)
+ }
+ }
+ }
+}
+wrapper.finalizedBy bootstrapWrapperProperties
+
+// Remove the generated batch file since we don't test building in the Windows environment.
+task removeWindowsScript(type: Delete) {
+ delete "$rootDir/gradlew.bat"
+}
+wrapper.finalizedBy removeWindowsScript