You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by ch...@apache.org on 2019/05/02 08:42:21 UTC
[flink] branch release-1.8 updated (633e78a -> dc857f3)
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a change to branch release-1.8
in repository https://gitbox.apache.org/repos/asf/flink.git.
from 633e78a [hotfix][travis] Fix scala 2.12 profile name
new 9109b58 [hotfix][travis] Add sleep before timestamp update
new faad0ae [FLINK-12346][travis][build] Account for timestamps in scala-suffix check
new 6257631 [hotfix][travis] Remove unnecessary faraday installation
new 58b46aa [hotfix][travis] Don't cache gems
new afa93bd [hotfix][travis] Exclude all jars from cache
new a2d7050 [hotfix][travis] Speed up dependency convergence
new fa4b0d0 [hotfix][travis] Allow scala-suffix check to download plugins
new 66a08d9 [hotfix][travis] Speed up scala-suffix check
new 3f1f7b9 [hotfix][travis] Move docker wordcount to nightly tests
new dc857f3 [hotfix][travis] Install kubernetes/docker only in E2E profiles
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.travis.yml | 18 -----
flink-end-to-end-tests/run-pre-commit-tests.sh | 1 -
pom.xml | 3 +-
tools/check_dependency_convergence.sh | 79 ----------------------
tools/travis/nightly.sh | 3 +
.../travis/setup_docker.sh | 7 +-
tools/travis/setup_kubernetes.sh | 24 +++++++
tools/travis/splits/split_container.sh | 2 +-
tools/travis_controller.sh | 23 ++-----
tools/verify_scala_suffixes.sh | 15 ++--
10 files changed, 49 insertions(+), 126 deletions(-)
delete mode 100755 tools/check_dependency_convergence.sh
copy flink-connectors/flink-connector-kafka-0.8/src/main/resources/META-INF/services/org.apache.flink.table.factories.TableFactory => tools/travis/setup_docker.sh (71%)
mode change 100644 => 100755
create mode 100755 tools/travis/setup_kubernetes.sh
[flink] 03/10: [hotfix][travis] Remove unnecessary faraday
installation
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.8
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 6257631b547d6a538b768f846a43708906f3d65a
Author: Chesnay Schepler <ch...@apache.org>
AuthorDate: Sat Apr 27 10:32:53 2019 +0200
[hotfix][travis] Remove unnecessary faraday installation
---
.travis.yml | 3 ---
1 file changed, 3 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 2489c25..3dd0d8b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -13,8 +13,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# s3 deployment based on http://about.travis-ci.org/blog/2012-12-18-travis-artifacts/
-
# send to fully-virtualized infrastructure: https://docs.travis-ci.com/user/trusty-ci-environment/
sudo: required
dist: trusty
@@ -48,7 +46,6 @@ env:
- CHANGE_MINIKUBE_NONE_USER=true
before_script:
- - "gem install --no-document --version 0.8.9 faraday "
- "export -f travis_nanoseconds"
- "export -f travis_time_start"
- "export -f travis_time_finish"
[flink] 01/10: [hotfix][travis] Add sleep before timestamp update
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.8
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 9109b58474a2936c1aa2f7e3358ae55b0c830bc2
Author: Chesnay Schepler <ch...@apache.org>
AuthorDate: Sun Apr 28 12:20:56 2019 +0200
[hotfix][travis] Add sleep before timestamp update
---
tools/travis_controller.sh | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/travis_controller.sh b/tools/travis_controller.sh
index 39ada67..8d8624a 100755
--- a/tools/travis_controller.sh
+++ b/tools/travis_controller.sh
@@ -156,6 +156,8 @@ elif [ $STAGE != "$STAGE_CLEANUP" ]; then
# adjust timestamps to prevent recompilation
find . -type f -name '*.java' | xargs touch
find . -type f -name '*.scala' | xargs touch
+ # wait a bit for better odds of different timestamps
+ sleep 5
find . -type f -name '*.class' | xargs touch
find . -type f -name '*.timestamp' | xargs touch
travis_time_finish
[flink] 07/10: [hotfix][travis] Allow scala-suffix check to
download plugins
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.8
in repository https://gitbox.apache.org/repos/asf/flink.git
commit fa4b0d0eb1670076215f125e9ada2fdc22304150
Author: Chesnay Schepler <ch...@apache.org>
AuthorDate: Thu Apr 11 12:52:35 2019 +0200
[hotfix][travis] Allow scala-suffix check to download plugins
---
tools/verify_scala_suffixes.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/verify_scala_suffixes.sh b/tools/verify_scala_suffixes.sh
index 0f94c2d..c20a607 100755
--- a/tools/verify_scala_suffixes.sh
+++ b/tools/verify_scala_suffixes.sh
@@ -116,7 +116,7 @@ while read line; do
block_infected=1
fi
fi
-done < <(mvn -o dependency:tree -Dincludes=org.scala-lang -pl ${excluded_modules} | tee /dev/tty)
+done < <(mvn -nsu dependency:tree -Dincludes=org.scala-lang -pl ${excluded_modules} | tee /dev/tty)
# deduplicate and sort
[flink] 05/10: [hotfix][travis] Exclude all jars from cache
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.8
in repository https://gitbox.apache.org/repos/asf/flink.git
commit afa93bda7d8237aa0a1c71588983955251950c8a
Author: Chesnay Schepler <ch...@apache.org>
AuthorDate: Fri Apr 26 13:49:41 2019 +0200
[hotfix][travis] Exclude all jars from cache
Jar caching is not required since they are rebuilt in the test profiles anyway.
---
tools/travis_controller.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/travis_controller.sh b/tools/travis_controller.sh
index 8d8624a..33d7961 100755
--- a/tools/travis_controller.sh
+++ b/tools/travis_controller.sh
@@ -119,8 +119,8 @@ if [ $STAGE == "$STAGE_COMPILE" ]; then
# the packing&upload / download&unpacking process
# by removing files not required for subsequent stages
- # original jars
- find "$CACHE_FLINK_DIR" -maxdepth 8 -type f -name 'original-*.jar' | xargs rm -rf
+ # jars are re-built in subsequent stages, so no need to cache them (cannot be avoided)
+ find "$CACHE_FLINK_DIR" -maxdepth 8 -type f -name '*.jar' | xargs rm -rf
# .git directory
# not deleting this can cause build stability issues
[flink] 06/10: [hotfix][travis] Speed up dependency convergence
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.8
in repository https://gitbox.apache.org/repos/asf/flink.git
commit a2d705097ca1e1a11a700797ed64fc0a1bb4475c
Author: Chesnay Schepler <ch...@apache.org>
AuthorDate: Fri Apr 26 20:31:32 2019 +0200
[hotfix][travis] Speed up dependency convergence
Run dependency convergence in main compile run.
Invoking maven once per modules requires significant time.
Run convergence in install phase (i.e. after the shade plugin) to work against dependency-reduced poms.
---
pom.xml | 3 +-
tools/check_dependency_convergence.sh | 79 -----------------------------------
tools/travis_controller.sh | 15 +------
3 files changed, 3 insertions(+), 94 deletions(-)
diff --git a/pom.xml b/pom.xml
index e2b5737..6a9f52c 100644
--- a/pom.xml
+++ b/pom.xml
@@ -128,6 +128,7 @@ under the License.
<powermock.version>2.0.0-RC.4</powermock.version>
<hamcrest.version>1.3</hamcrest.version>
<japicmp.skip>false</japicmp.skip>
+ <flink.convergence.phase>validate</flink.convergence.phase>
<!--
Keeping the MiniKDC version fixed instead of taking hadoop version dependency
to support testing Kafka, ZK etc., modules that does not have Hadoop dependency
@@ -866,7 +867,7 @@ under the License.
<executions>
<execution>
<id>dependency-convergence</id>
- <phase>validate</phase>
+ <phase>${flink.convergence.phase}</phase>
</execution>
</executions>
</plugin>
diff --git a/tools/check_dependency_convergence.sh b/tools/check_dependency_convergence.sh
deleted file mode 100755
index a9294b8..0000000
--- a/tools/check_dependency_convergence.sh
+++ /dev/null
@@ -1,79 +0,0 @@
-#!/usr/bin/env bash
-################################################################################
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-################################################################################
-
-HERE="`dirname \"$0\"`" # relative
-HERE="`( cd \"$HERE\" && pwd )`" # absolutized and normalized
-if [ -z "$HERE" ] ; then
- # error; for some reason, the path is not accessible
- # to the script (e.g. permissions re-evaled after suid)
- exit 1 # fail
-fi
-
-FLINK_DIR=HERE
-
-if [[ $(basename ${HERE}) == "tools" ]] ; then
- FLINK_DIR="${HERE}/.."
-fi
-
-FLINK_DIR="`( cd \"${FLINK_DIR}\" && pwd )`"
-
-echo ${FLINK_DIR}
-
-# get list of all flink modules
-# searches for directories containing a pom.xml file
-# sorts the list alphabetically
-# only accepts directories starting with "flink" to filter force-shading
-modules=$(find . -maxdepth 3 -name 'pom.xml' -printf '%h\n' | sort -u | grep "flink")
-
-for module in ${modules}
-do
- # There are no Scala 2.12 dependencies for older Kafka versions
- if [[ $PROFILE == *"scala-2.12"* ]]; then
- if [[ $module == *"kafka-0.8"* ]]; then
- echo "excluding ${module} because we build for Scala 2.12"
- continue 2
- fi
- if [[ $module == *"kafka-0.9"* ]]; then
- echo "excluding ${module} because we build for Scala 2.12"
- continue 2
- fi
- fi
-
- # we are only interested in child modules
- for other_module in ${modules}
- do
- if [[ "${other_module}" != "${module}" && "${other_module}" = "${module}"/* ]]; then
- echo "excluding ${module} since it is not a leaf module"
- continue 2
- fi
- done
-
- cd "${module}"
- echo "checking ${module}"
- output=$(mvn validate $PROFILE -nsu -Dcheckstyle.skip=true -Dcheck-convergence)
- exit_code=$?
- if [[ ${exit_code} != 0 ]]; then
- echo "dependency convergence failed."
- echo "${output}"
- exit ${exit_code}
- fi
- cd "${FLINK_DIR}"
-done
-
-exit 0
diff --git a/tools/travis_controller.sh b/tools/travis_controller.sh
index 33d7961..30b3923 100755
--- a/tools/travis_controller.sh
+++ b/tools/travis_controller.sh
@@ -59,7 +59,7 @@ EXIT_CODE=0
# Run actual compile&test steps
if [ $STAGE == "$STAGE_COMPILE" ]; then
- MVN="mvn clean install -nsu -Dflink.forkCount=2 -Dflink.forkCountTestPackage=2 -Dmaven.javadoc.skip=true -B -DskipTests $PROFILE"
+ MVN="mvn clean install -nsu -Dflink.convergence.phase=install -Pcheck-convergence -Dflink.forkCount=2 -Dflink.forkCountTestPackage=2 -Dmaven.javadoc.skip=true -B -DskipTests $PROFILE"
$MVN
EXIT_CODE=$?
@@ -75,19 +75,6 @@ if [ $STAGE == "$STAGE_COMPILE" ]; then
printf "Previous build failure detected, skipping scala-suffixes check.\n"
printf "==============================================================================\n"
fi
-
- if [ $EXIT_CODE == 0 ]; then
- printf "\n\n==============================================================================\n"
- printf "Checking dependency convergence\n"
- printf "==============================================================================\n"
-
- ./tools/check_dependency_convergence.sh
- EXIT_CODE=$?
- else
- printf "\n==============================================================================\n"
- printf "Previous build failure detected, skipping dependency-convergence check.\n"
- printf "==============================================================================\n"
- fi
if [ $EXIT_CODE == 0 ]; then
check_shaded_artifacts
[flink] 02/10: [FLINK-12346][travis][build] Account for timestamps
in scala-suffix check
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.8
in repository https://gitbox.apache.org/repos/asf/flink.git
commit faad0aecd5d28df54533e9d0ef7c7493e63757f2
Author: Chesnay Schepler <ch...@apache.org>
AuthorDate: Sat Apr 27 13:18:45 2019 +0200
[FLINK-12346][travis][build] Account for timestamps in scala-suffix check
---
tools/verify_scala_suffixes.sh | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tools/verify_scala_suffixes.sh b/tools/verify_scala_suffixes.sh
index e4099de..0f94c2d 100755
--- a/tools/verify_scala_suffixes.sh
+++ b/tools/verify_scala_suffixes.sh
@@ -93,14 +93,14 @@ infected=""
clean=""
while read line; do
- if [[ $line == "$BEGIN"* ]]; then
+ if [[ $line == *"$BEGIN"* ]]; then
reached_block=1
# Maven module name
block_name=`[[ "$line" =~ .*(flink-?[-a-zA-Z0-9.]*).* ]] && echo ${BASH_REMATCH[1]}`
- elif [[ $line == "$SEPARATOR" ]] && [[ $reached_block -eq 1 ]]; then
+ elif [[ $line == *"$SEPARATOR"* ]] && [[ $reached_block -eq 1 ]]; then
reached_block=0
in_block=1
- elif [[ $line == "$END" ]] && [[ $in_block -eq 1 ]]; then
+ elif [[ $line == *"$END"* ]] && [[ $in_block -eq 1 ]]; then
if [[ $block_infected -eq 0 ]]; then
clean="$block_name $clean"
fi
[flink] 10/10: [hotfix][travis] Install kubernetes/docker only in
E2E profiles
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.8
in repository https://gitbox.apache.org/repos/asf/flink.git
commit dc857f3fb6d93c1e50672a6567d337e57b1f2522
Author: Chesnay Schepler <ch...@apache.org>
AuthorDate: Fri Apr 26 12:44:05 2019 +0200
[hotfix][travis] Install kubernetes/docker only in E2E profiles
---
.travis.yml | 14 --------------
tools/travis/nightly.sh | 3 +++
tools/travis/setup_docker.sh | 19 +++++++++++++++++++
tools/travis/setup_kubernetes.sh | 24 ++++++++++++++++++++++++
4 files changed, 46 insertions(+), 14 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 8bf948c..09f2024 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -48,24 +48,10 @@ before_script:
- "export -f travis_nanoseconds"
- "export -f travis_time_start"
- "export -f travis_time_finish"
- # Download kubectl, which is a requirement for using minikube.
- - curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
- # Download minikube.
- - curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.25.2/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
- - sudo minikube start --vm-driver=none --kubernetes-version=v1.9.0
- # Fix the kubectl context, as it's often stale.
- - minikube update-context
- # Wait for Kubernetes to be up and ready.
- - JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.type}={@.status};{end}{end}'; until kubectl get nodes -o jsonpath="$JSONPATH" 2>&1 | grep -q "Ready=True"; do sleep 1; done
# Install maven 3.2.5 since trusty uses 3.3.9 for which shading is broken
before_install:
- source ./tools/travis/setup_maven.sh
-# Installing the specified docker compose version
- - sudo rm /usr/local/bin/docker-compose
- - curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
- - chmod +x docker-compose
- - sudo mv docker-compose /usr/local/bin
notifications:
slack:
diff --git a/tools/travis/nightly.sh b/tools/travis/nightly.sh
index c742931..276274d 100755
--- a/tools/travis/nightly.sh
+++ b/tools/travis/nightly.sh
@@ -27,6 +27,9 @@ fi
SCRIPT=$1
+source ${HERE}/setup_docker.sh
+source ${HERE}/setup_kubernetes.sh
+
ARTIFACTS_DIR="${HERE}/artifacts"
mkdir -p $ARTIFACTS_DIR || { echo "FAILURE: cannot create log directory '${ARTIFACTS_DIR}'." ; exit 1; }
diff --git a/tools/travis/setup_docker.sh b/tools/travis/setup_docker.sh
new file mode 100755
index 0000000..6279384
--- /dev/null
+++ b/tools/travis/setup_docker.sh
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+sudo rm /usr/local/bin/docker-compose
+curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
+chmod +x docker-compose
+sudo mv docker-compose /usr/local/bin
diff --git a/tools/travis/setup_kubernetes.sh b/tools/travis/setup_kubernetes.sh
new file mode 100755
index 0000000..e84ab6e
--- /dev/null
+++ b/tools/travis/setup_kubernetes.sh
@@ -0,0 +1,24 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Download kubectl, which is a requirement for using minikube.
+curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
+# Download minikube.
+curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.25.2/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
+sudo minikube start --vm-driver=none --kubernetes-version=v1.9.0
+# Fix the kubectl context, as it's often stale.
+minikube update-context
+# Wait for Kubernetes to be up and ready.
+JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.type}={@.status};{end}{end}'; until kubectl get nodes -o jsonpath="$JSONPATH" 2>&1 | grep -q "Ready=True"; do sleep 1; done
[flink] 04/10: [hotfix][travis] Don't cache gems
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.8
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 58b46aa8c3a4fe0d3c4d54300b0aa756a665343c
Author: Chesnay Schepler <ch...@apache.org>
AuthorDate: Thu May 2 10:37:33 2019 +0200
[hotfix][travis] Don't cache gems
---
.travis.yml | 1 -
1 file changed, 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index 3dd0d8b..8bf948c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -21,7 +21,6 @@ cache:
# default timeout is too low
timeout: 600
directories:
- - $HOME/.rvm/
- $HOME/.m2
- $HOME/flink_cache
# keep in sync with tools/travis/setup_maven.sh
[flink] 08/10: [hotfix][travis] Speed up scala-suffix check
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.8
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 66a08d9a21618f22c608c7686ef3648da35a4e95
Author: Chesnay Schepler <ch...@apache.org>
AuthorDate: Fri Apr 26 13:28:04 2019 +0200
[hotfix][travis] Speed up scala-suffix check
- fix find -mindepth parameter
- pass PROFILE to maven to prevent downloads of modules that weren't built beforehand
- add -maxdepth parameter for pom.xml searches
---
tools/travis_controller.sh | 2 +-
tools/verify_scala_suffixes.sh | 9 +++++----
2 files changed, 6 insertions(+), 5 deletions(-)
diff --git a/tools/travis_controller.sh b/tools/travis_controller.sh
index 30b3923..c0f70a5 100755
--- a/tools/travis_controller.sh
+++ b/tools/travis_controller.sh
@@ -68,7 +68,7 @@ if [ $STAGE == "$STAGE_COMPILE" ]; then
printf "Checking scala suffixes\n"
printf "==============================================================================\n"
- ./tools/verify_scala_suffixes.sh
+ ./tools/verify_scala_suffixes.sh "${PROFILE}"
EXIT_CODE=$?
else
printf "\n==============================================================================\n"
diff --git a/tools/verify_scala_suffixes.sh b/tools/verify_scala_suffixes.sh
index c20a607..2123ab1 100755
--- a/tools/verify_scala_suffixes.sh
+++ b/tools/verify_scala_suffixes.sh
@@ -61,6 +61,7 @@
# [INFO] \- org.scala-lang:scala-library:jar:2.10.4:compile
# [INFO]
+MAVEN_ARGUMENTS=${1:-""}
if [[ `basename $PWD` == "tools" ]] ; then
cd ..
@@ -83,7 +84,7 @@ block_infected=0
# a) are not deployed during a release
# b) exist only for dev purposes
# c) no-one should depend on them
-e2e_modules=$(find flink-end-to-end-tests -mindeptha 2 -maxdepth 5 -name 'pom.xml' -printf '%h\n' | sort -u | tr '\n' ',')
+e2e_modules=$(find flink-end-to-end-tests -mindepth 2 -maxdepth 5 -name 'pom.xml' -printf '%h\n' | sort -u | tr '\n' ',')
excluded_modules=\!${e2e_modules//,/,\!},!flink-docs
echo "Analyzing modules for Scala dependencies using 'mvn dependency:tree'."
@@ -116,7 +117,7 @@ while read line; do
block_infected=1
fi
fi
-done < <(mvn -nsu dependency:tree -Dincludes=org.scala-lang -pl ${excluded_modules} | tee /dev/tty)
+done < <(mvn -nsu dependency:tree -Dincludes=org.scala-lang -pl ${excluded_modules} ${MAVEN_ARGUMENTS} | tee /dev/tty)
# deduplicate and sort
@@ -140,7 +141,7 @@ echo
echo "Checking Scala-free modules:"
for module in $clean; do
- out=`find . -name 'pom.xml' -not -path '*target*' -exec grep "${module}_\d\+\.\d\+</artifactId>" "{}" \;`
+ out=`find . -maxdepth 3 -name 'pom.xml' -not -path '*target*' -exec grep "${module}_\d\+\.\d\+</artifactId>" "{}" \;`
if [[ "$out" == "" ]]; then
printf "$GREEN OK $NC $module\n"
else
@@ -154,7 +155,7 @@ echo
echo "Checking Scala-dependent modules:"
for module in $infected; do
- out=`find . -name 'pom.xml' -not -path '*target*' -exec grep "${module}</artifactId>" "{}" \;`
+ out=`find . -maxdepth 3 -name 'pom.xml' -not -path '*target*' -exec grep "${module}</artifactId>" "{}" \;`
if [[ "$out" == "" ]]; then
printf "$GREEN OK $NC $module\n"
else
[flink] 09/10: [hotfix][travis] Move docker wordcount to nightly
tests
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.8
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 3f1f7b96068c4586503bac34edbca1e0b5d18c1f
Author: Chesnay Schepler <ch...@apache.org>
AuthorDate: Sat Apr 27 18:26:44 2019 +0200
[hotfix][travis] Move docker wordcount to nightly tests
---
flink-end-to-end-tests/run-pre-commit-tests.sh | 1 -
tools/travis/splits/split_container.sh | 2 +-
2 files changed, 1 insertion(+), 2 deletions(-)
diff --git a/flink-end-to-end-tests/run-pre-commit-tests.sh b/flink-end-to-end-tests/run-pre-commit-tests.sh
index f40fd56..3b81b3f 100755
--- a/flink-end-to-end-tests/run-pre-commit-tests.sh
+++ b/flink-end-to-end-tests/run-pre-commit-tests.sh
@@ -61,7 +61,6 @@ run_test "class loading end-to-end test" "$END_TO_END_DIR/test-scripts/test_stre
run_test "Shaded Hadoop S3A end-to-end test" "$END_TO_END_DIR/test-scripts/test_shaded_hadoop_s3a.sh"
run_test "Shaded Presto S3 end-to-end test" "$END_TO_END_DIR/test-scripts/test_shaded_presto_s3.sh"
run_test "Distributed cache end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_distributed_cache_via_blob.sh"
-run_test "Wordcount end-to-end test in docker env" "$END_TO_END_DIR/test-scripts/test_docker_embedded_job.sh"
printf "\n[PASS] All tests passed\n"
exit 0
diff --git a/tools/travis/splits/split_container.sh b/tools/travis/splits/split_container.sh
index c62905f..d34d55e 100755
--- a/tools/travis/splits/split_container.sh
+++ b/tools/travis/splits/split_container.sh
@@ -43,8 +43,8 @@ echo "Flink distribution directory: $FLINK_DIR"
# run_test "<description>" "$END_TO_END_DIR/test-scripts/<script_name>"
+run_test "Wordcount on Docker test" "$END_TO_END_DIR/test-scripts/test_docker_embedded_job.sh"
run_test "Running Kerberized YARN on Docker test " "$END_TO_END_DIR/test-scripts/test_yarn_kerberos_docker.sh"
-
run_test "Run kubernetes test" "$END_TO_END_DIR/test-scripts/test_kubernetes_embedded_job.sh"
printf "\n[PASS] All tests passed\n"