You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by tw...@apache.org on 2018/11/14 12:20:29 UTC
[flink] branch cron-1.7-e2e updated (fe4b5bd -> b6d97cb)
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a change to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git.
from fe4b5bd Set branch to release-1.7
new 4756ccf Integrate scala 2.12 e2e tests
new 32a9fd1 Integrate hadoopfree e2e tests
new 8e8aa87 add executable permission
new 83802f6 [hotfix] Add missing end-to-end tests
new c31291e [hotfix] Remove pre-commit tests from nightly tests
new 31e8d61 [hotfix] Remove duplicate test execution
new fc023e6 [hotfix] Change splits because of Travis time constraints
new 01340fe [hotfix] Enable parallel building
new 5a054b4 [hotfix] Add ES 6 end-to-end test again
new 9ec4ca4 [hotfix] Update Hadoop-free tests
new b6d97cb [hotfix] Reduce test matrix for releases
The 11 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.travis.yml | 4 ++--
nightly.sh | 2 +-
splits/split_checkpoints.sh | 9 ++++++---
splits/{split_docker_e2e.sh => split_container.sh} | 2 ++
splits/{split_kubernetes_e2e.sh => split_heavy.sh} | 5 +++--
splits/split_misc.sh | 6 +++++-
splits/{split_misc.sh => split_misc_hadoopfree.sh} | 7 ++++---
7 files changed, 23 insertions(+), 12 deletions(-)
rename splits/{split_docker_e2e.sh => split_container.sh} (95%)
rename splits/{split_kubernetes_e2e.sh => split_heavy.sh} (87%)
copy splits/{split_misc.sh => split_misc_hadoopfree.sh} (91%)
[flink] 02/11: Integrate hadoopfree e2e tests
Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 32a9fd1094a7cc8042017aa92547523ca87b208d
Author: zentol <ch...@apache.org>
AuthorDate: Wed Nov 7 09:22:39 2018 +0100
Integrate hadoopfree e2e tests
---
.travis.yml | 30 ++++++++++++++++++
splits/split_misc_hadoopfree.sh | 68 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 98 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index 13e504a..e2f9ae4 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -105,6 +105,36 @@ matrix:
- BRANCH="master"
- PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
- SCRIPT="split_kubernetes_e2e.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-DwithoutHadoop -De2e-metrics"
+ - SCRIPT="split_misc_hadoopfree.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-DwithoutHadoop"
+ - SCRIPT="split_ha.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-DwithoutHadoop"
+ - SCRIPT="split_sticky.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-DwithoutHadoop"
+ - SCRIPT="split_checkpoints.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-DwithoutHadoop"
+ - SCRIPT="split_docker_e2e.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-DwithoutHadoop"
+ - SCRIPT="split_kubernetes_e2e.sh"
git:
depth: 100
diff --git a/splits/split_misc_hadoopfree.sh b/splits/split_misc_hadoopfree.sh
new file mode 100644
index 0000000..3d66896
--- /dev/null
+++ b/splits/split_misc_hadoopfree.sh
@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+END_TO_END_DIR="`dirname \"$0\"`" # relative
+END_TO_END_DIR="`( cd \"$END_TO_END_DIR\" && pwd -P)`" # absolutized and normalized
+if [ -z "$END_TO_END_DIR" ] ; then
+ # error; for some reason, the path is not accessible
+ # to the script (e.g. permissions re-evaled after suid)
+ exit 1 # fail
+fi
+
+export END_TO_END_DIR
+
+if [ -z "$FLINK_DIR" ] ; then
+ echo "You have to export the Flink distribution directory as FLINK_DIR"
+ exit 1
+fi
+
+source "${END_TO_END_DIR}/test-scripts/test-runner-common.sh"
+
+FLINK_DIR="`( cd \"$FLINK_DIR\" && pwd -P)`" # absolutized and normalized
+
+echo "flink-end-to-end-test directory: $END_TO_END_DIR"
+echo "Flink distribution directory: $FLINK_DIR"
+
+# Template for adding a test:
+
+# run_test "<description>" "$END_TO_END_DIR/test-scripts/<script_name>"
+
+run_test "Queryable state (rocksdb) end-to-end test" "$END_TO_END_DIR/test-scripts/test_queryable_state.sh rocksdb"
+run_test "Queryable state (rocksdb) with TM restart end-to-end test" "$END_TO_END_DIR/test-scripts/test_queryable_state_restart_tm.sh" "skip_check_exceptions"
+
+run_test "DataSet allround end-to-end test" "$END_TO_END_DIR/test-scripts/test_batch_allround.sh"
+run_test "Stateful stream job upgrade end-to-end test" "$END_TO_END_DIR/test-scripts/test_stateful_stream_job_upgrade.sh 2 4"
+
+run_test "Elasticsearch (v1.7.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 1 https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.tar.gz"
+run_test "Elasticsearch (v2.3.5) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 2 https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.3.5/elasticsearch-2.3.5.tar.gz"
+run_test "Elasticsearch (v5.1.2) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 5 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.2.tar.gz"
+run_test "Elasticsearch (v6.3.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 6 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz"
+
+run_test "Quickstarts Java nightly end-to-end test" "$END_TO_END_DIR/test-scripts/test_quickstarts.sh java"
+run_test "Quickstarts Scala nightly end-to-end test" "$END_TO_END_DIR/test-scripts/test_quickstarts.sh scala"
+
+run_test "Avro Confluent Schema Registry nightly end-to-end test" "$END_TO_END_DIR/test-scripts/test_confluent_schema_registry.sh"
+
+run_test "State TTL Heap backend end-to-end test" "$END_TO_END_DIR/test-scripts/test_stream_state_ttl.sh file"
+run_test "State TTL RocksDb backend end-to-end test" "$END_TO_END_DIR/test-scripts/test_stream_state_ttl.sh rocks"
+
+run_test "Run kubernetes test" "$END_TO_END_DIR/test-scripts/test_kubernetes_embedded_job.sh"
+
+printf "\n[PASS] All tests passed\n"
+exit 0
[flink] 10/11: [hotfix] Update Hadoop-free tests
Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 9ec4ca435ee9cf1bfbadf081d39360a6761400bb
Author: Timo Walther <tw...@apache.org>
AuthorDate: Mon Nov 12 09:42:37 2018 +0100
[hotfix] Update Hadoop-free tests
---
splits/split_misc_hadoopfree.sh | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/splits/split_misc_hadoopfree.sh b/splits/split_misc_hadoopfree.sh
index 8281300..6c552bd 100755
--- a/splits/split_misc_hadoopfree.sh
+++ b/splits/split_misc_hadoopfree.sh
@@ -43,17 +43,21 @@ echo "Flink distribution directory: $FLINK_DIR"
# run_test "<description>" "$END_TO_END_DIR/test-scripts/<script_name>"
+run_test "Flink CLI end-to-end test" "$END_TO_END_DIR/test-scripts/test_cli.sh"
+
run_test "Queryable state (rocksdb) end-to-end test" "$END_TO_END_DIR/test-scripts/test_queryable_state.sh rocksdb"
run_test "Queryable state (rocksdb) with TM restart end-to-end test" "$END_TO_END_DIR/test-scripts/test_queryable_state_restart_tm.sh" "skip_check_exceptions"
run_test "DataSet allround end-to-end test" "$END_TO_END_DIR/test-scripts/test_batch_allround.sh"
+run_test "Streaming SQL end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_sql.sh" "skip_check_exceptions"
+run_test "Streaming File Sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_file_sink.sh" "skip_check_exceptions"
+run_test "Streaming File Sink s3 end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_file_sink.sh s3" "skip_check_exceptions"
run_test "Stateful stream job upgrade end-to-end test" "$END_TO_END_DIR/test-scripts/test_stateful_stream_job_upgrade.sh 2 4"
run_test "Elasticsearch (v1.7.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 1 https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.tar.gz"
run_test "Elasticsearch (v2.3.5) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 2 https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.3.5/elasticsearch-2.3.5.tar.gz"
run_test "Elasticsearch (v5.1.2) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 5 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.2.tar.gz"
-# Disabled because of test instability, see FLINK-10361
-#run_test "Elasticsearch (v6.3.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 6 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz"
+run_test "Elasticsearch (v6.3.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 6 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz"
run_test "Quickstarts Java nightly end-to-end test" "$END_TO_END_DIR/test-scripts/test_quickstarts.sh java"
run_test "Quickstarts Scala nightly end-to-end test" "$END_TO_END_DIR/test-scripts/test_quickstarts.sh scala"
@@ -63,7 +67,5 @@ run_test "Avro Confluent Schema Registry nightly end-to-end test" "$END_TO_END_D
run_test "State TTL Heap backend end-to-end test" "$END_TO_END_DIR/test-scripts/test_stream_state_ttl.sh file"
run_test "State TTL RocksDb backend end-to-end test" "$END_TO_END_DIR/test-scripts/test_stream_state_ttl.sh rocks"
-run_test "Run kubernetes test" "$END_TO_END_DIR/test-scripts/test_kubernetes_embedded_job.sh"
-
printf "\n[PASS] All tests passed\n"
exit 0
[flink] 11/11: [hotfix] Reduce test matrix for releases
Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git
commit b6d97cb95795ae21b3a9332c9d6caff789e2c98e
Author: Timo Walther <tw...@apache.org>
AuthorDate: Wed Nov 14 13:14:42 2018 +0100
[hotfix] Reduce test matrix for releases
For simplification of the cron jobs we don't test dependencies of releases
thus we don't test the full Scala/Hadoop matrix.
---
.travis.yml | 60 ------------------------------------------------------------
1 file changed, 60 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 7b98f7a..3cbecfe 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -75,66 +75,6 @@ matrix:
- BRANCH="release-1.7"
- PROFILE="-Dhadoop.version=2.8.3"
- SCRIPT="split_heavy.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12 -De2e-metrics"
- - SCRIPT="split_misc.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
- - SCRIPT="split_ha.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
- - SCRIPT="split_sticky.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
- - SCRIPT="split_checkpoints.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
- - SCRIPT="split_container.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
- - SCRIPT="split_heavy.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-DwithoutHadoop -De2e-metrics"
- - SCRIPT="split_misc_hadoopfree.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-DwithoutHadoop"
- - SCRIPT="split_ha.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-DwithoutHadoop"
- - SCRIPT="split_sticky.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-DwithoutHadoop"
- - SCRIPT="split_checkpoints.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-DwithoutHadoop"
- - SCRIPT="split_container.sh"
- - env:
- - REMOTE="apache"
- - BRANCH="master"
- - PROFILE="-DwithoutHadoop"
- - SCRIPT="split_heavy.sh"
git:
depth: 100
[flink] 09/11: [hotfix] Add ES 6 end-to-end test again
Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 5a054b4c82e071e1267432a534e7d3dd222f2da8
Author: Timo Walther <tw...@apache.org>
AuthorDate: Thu Nov 8 14:03:41 2018 +0100
[hotfix] Add ES 6 end-to-end test again
---
splits/split_misc.sh | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/splits/split_misc.sh b/splits/split_misc.sh
index 62d4b34..3f022b4 100755
--- a/splits/split_misc.sh
+++ b/splits/split_misc.sh
@@ -58,8 +58,7 @@ run_test "Stateful stream job upgrade end-to-end test" "$END_TO_END_DIR/test-scr
run_test "Elasticsearch (v1.7.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 1 https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.tar.gz"
run_test "Elasticsearch (v2.3.5) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 2 https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.3.5/elasticsearch-2.3.5.tar.gz"
run_test "Elasticsearch (v5.1.2) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 5 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.2.tar.gz"
-# Disabled because of test instability, see FLINK-10361
-#run_test "Elasticsearch (v6.3.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 6 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz"
+run_test "Elasticsearch (v6.3.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 6 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz"
run_test "Quickstarts Java nightly end-to-end test" "$END_TO_END_DIR/test-scripts/test_quickstarts.sh java"
run_test "Quickstarts Scala nightly end-to-end test" "$END_TO_END_DIR/test-scripts/test_quickstarts.sh scala"
[flink] 04/11: [hotfix] Add missing end-to-end tests
Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 83802f6d6d23973088c2d474606a7b544654ce4e
Author: Timo Walther <tw...@apache.org>
AuthorDate: Wed Nov 7 09:41:52 2018 +0100
[hotfix] Add missing end-to-end tests
---
splits/split_checkpoints.sh | 9 ++++++---
splits/split_misc.sh | 10 ++++++++++
2 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/splits/split_checkpoints.sh b/splits/split_checkpoints.sh
index 930b5a4..5dfe7e3 100755
--- a/splits/split_checkpoints.sh
+++ b/splits/split_checkpoints.sh
@@ -49,9 +49,12 @@ run_test "Resuming Savepoint (file, async, scale up) end-to-end test" "$END_TO_E
run_test "Resuming Savepoint (file, sync, scale up) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 2 4 file false"
run_test "Resuming Savepoint (file, async, scale down) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 4 2 file true"
run_test "Resuming Savepoint (file, sync, scale down) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 4 2 file false"
-run_test "Resuming Savepoint (rocks, no parallelism change) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 2 2 rocks"
-run_test "Resuming Savepoint (rocks, scale up) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 2 4 rocks"
-run_test "Resuming Savepoint (rocks, scale down) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 4 2 rocks"
+run_test "Resuming Savepoint (rocks, no parallelism change, heap timers) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 2 2 rocks false heap"
+run_test "Resuming Savepoint (rocks, scale up, heap timers) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 2 4 rocks false heap"
+run_test "Resuming Savepoint (rocks, scale down, heap timers) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 4 2 rocks false heap"
+run_test "Resuming Savepoint (rocks, no parallelism change, rocks timers) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 2 2 rocks false rocks"
+run_test "Resuming Savepoint (rocks, scale up, rocks timers) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 2 4 rocks false rocks"
+run_test "Resuming Savepoint (rocks, scale down, rocks timers) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 4 2 rocks false rocks"
run_test "Resuming Externalized Checkpoint (file, async, no parallelism change) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 2 2 file true true"
run_test "Resuming Externalized Checkpoint (file, sync, no parallelism change) end-to-end test" "$END_TO_END_DIR/test-scripts/test_resume_externalized_checkpoints.sh 2 2 file false true"
diff --git a/splits/split_misc.sh b/splits/split_misc.sh
index 8050364..a9ef821 100755
--- a/splits/split_misc.sh
+++ b/splits/split_misc.sh
@@ -43,12 +43,16 @@ echo "Flink distribution directory: $FLINK_DIR"
# run_test "<description>" "$END_TO_END_DIR/test-scripts/<script_name>"
+run_test "Flink CLI end-to-end test" "$END_TO_END_DIR/test-scripts/test_cli.sh"
+
run_test "Queryable state (rocksdb) end-to-end test" "$END_TO_END_DIR/test-scripts/test_queryable_state.sh rocksdb"
run_test "Queryable state (rocksdb) with TM restart end-to-end test" "$END_TO_END_DIR/test-scripts/test_queryable_state_restart_tm.sh" "skip_check_exceptions"
run_test "DataSet allround end-to-end test" "$END_TO_END_DIR/test-scripts/test_batch_allround.sh"
run_test "Streaming SQL end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_sql.sh" "skip_check_exceptions"
run_test "Streaming bucketing end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_bucketing.sh" "skip_check_exceptions"
+run_test "Streaming File Sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_file_sink.sh" "skip_check_exceptions"
+run_test "Streaming File Sink s3 end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_file_sink.sh s3" "skip_check_exceptions"
run_test "Stateful stream job upgrade end-to-end test" "$END_TO_END_DIR/test-scripts/test_stateful_stream_job_upgrade.sh 2 4"
run_test "Elasticsearch (v1.7.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 1 https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.tar.gz"
@@ -66,5 +70,11 @@ run_test "State TTL RocksDb backend end-to-end test" "$END_TO_END_DIR/test-scrip
run_test "Run kubernetes test" "$END_TO_END_DIR/test-scripts/test_kubernetes_embedded_job.sh"
+run_test "SQL Client end-to-end test" "$END_TO_END_DIR/test-scripts/test_sql_client.sh"
+
+run_test "Heavy deployment end-to-end test" "$END_TO_END_DIR/test-scripts/test_heavy_deployment.sh"
+
+run_test "ConnectedComponents iterations with high parallelism end-to-end test" "$END_TO_END_DIR/test-scripts/test_high_parallelism_iterations.sh 25"
+
printf "\n[PASS] All tests passed\n"
exit 0
[flink] 05/11: [hotfix] Remove pre-commit tests from nightly tests
Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git
commit c31291eb8a1c40074af9d36fbbcdf548e55409cc
Author: Timo Walther <tw...@apache.org>
AuthorDate: Wed Nov 7 10:24:13 2018 +0100
[hotfix] Remove pre-commit tests from nightly tests
---
splits/split_kubernetes_e2e.sh | 1 -
1 file changed, 1 deletion(-)
diff --git a/splits/split_kubernetes_e2e.sh b/splits/split_kubernetes_e2e.sh
index 6e2a5b8..21c8e8d 100755
--- a/splits/split_kubernetes_e2e.sh
+++ b/splits/split_kubernetes_e2e.sh
@@ -44,7 +44,6 @@ echo "Flink distribution directory: $FLINK_DIR"
# run_test "<description>" "$END_TO_END_DIR/test-scripts/<script_name>"
run_test "Run kubernetes test" "$END_TO_END_DIR/test-scripts/test_kubernetes_embedded_job.sh"
-run_test "Run docker test" "$END_TO_END_DIR/test-scripts/test_docker_embedded_job.sh"
printf "\n[PASS] All tests passed\n"
exit 0
[flink] 07/11: [hotfix] Change splits because of Travis time
constraints
Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git
commit fc023e6d06c9859938934d75ac14635a313a4e2c
Author: Timo Walther <tw...@apache.org>
AuthorDate: Thu Nov 8 08:07:45 2018 +0100
[hotfix] Change splits because of Travis time constraints
---
.travis.yml | 12 ++++++------
splits/{split_docker_e2e.sh => split_container.sh} | 2 ++
splits/{split_kubernetes_e2e.sh => split_heavy.sh} | 4 +++-
splits/split_misc.sh | 7 ++-----
splits/split_misc_hadoopfree.sh | 3 ++-
5 files changed, 15 insertions(+), 13 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index e2f9ae4..7b98f7a 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -69,12 +69,12 @@ matrix:
- REMOTE="apache"
- BRANCH="release-1.7"
- PROFILE="-Dhadoop.version=2.8.3"
- - SCRIPT="split_docker_e2e.sh"
+ - SCRIPT="split_container.sh"
- env:
- REMOTE="apache"
- BRANCH="release-1.7"
- PROFILE="-Dhadoop.version=2.8.3"
- - SCRIPT="split_kubernetes_e2e.sh"
+ - SCRIPT="split_heavy.sh"
- env:
- REMOTE="apache"
- BRANCH="master"
@@ -99,12 +99,12 @@ matrix:
- REMOTE="apache"
- BRANCH="master"
- PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
- - SCRIPT="split_docker_e2e.sh"
+ - SCRIPT="split_container.sh"
- env:
- REMOTE="apache"
- BRANCH="master"
- PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
- - SCRIPT="split_kubernetes_e2e.sh"
+ - SCRIPT="split_heavy.sh"
- env:
- REMOTE="apache"
- BRANCH="master"
@@ -129,12 +129,12 @@ matrix:
- REMOTE="apache"
- BRANCH="master"
- PROFILE="-DwithoutHadoop"
- - SCRIPT="split_docker_e2e.sh"
+ - SCRIPT="split_container.sh"
- env:
- REMOTE="apache"
- BRANCH="master"
- PROFILE="-DwithoutHadoop"
- - SCRIPT="split_kubernetes_e2e.sh"
+ - SCRIPT="split_heavy.sh"
git:
depth: 100
diff --git a/splits/split_docker_e2e.sh b/splits/split_container.sh
similarity index 95%
rename from splits/split_docker_e2e.sh
rename to splits/split_container.sh
index ff0817b..c62905f 100755
--- a/splits/split_docker_e2e.sh
+++ b/splits/split_container.sh
@@ -45,5 +45,7 @@ echo "Flink distribution directory: $FLINK_DIR"
run_test "Running Kerberized YARN on Docker test " "$END_TO_END_DIR/test-scripts/test_yarn_kerberos_docker.sh"
+run_test "Run kubernetes test" "$END_TO_END_DIR/test-scripts/test_kubernetes_embedded_job.sh"
+
printf "\n[PASS] All tests passed\n"
exit 0
diff --git a/splits/split_kubernetes_e2e.sh b/splits/split_heavy.sh
similarity index 87%
rename from splits/split_kubernetes_e2e.sh
rename to splits/split_heavy.sh
index 21c8e8d..169ee31 100755
--- a/splits/split_kubernetes_e2e.sh
+++ b/splits/split_heavy.sh
@@ -43,7 +43,9 @@ echo "Flink distribution directory: $FLINK_DIR"
# run_test "<description>" "$END_TO_END_DIR/test-scripts/<script_name>"
-run_test "Run kubernetes test" "$END_TO_END_DIR/test-scripts/test_kubernetes_embedded_job.sh"
+run_test "Heavy deployment end-to-end test" "$END_TO_END_DIR/test-scripts/test_heavy_deployment.sh"
+
+run_test "ConnectedComponents iterations with high parallelism end-to-end test" "$END_TO_END_DIR/test-scripts/test_high_parallelism_iterations.sh 25"
printf "\n[PASS] All tests passed\n"
exit 0
diff --git a/splits/split_misc.sh b/splits/split_misc.sh
index abc1bcb..62d4b34 100755
--- a/splits/split_misc.sh
+++ b/splits/split_misc.sh
@@ -58,7 +58,8 @@ run_test "Stateful stream job upgrade end-to-end test" "$END_TO_END_DIR/test-scr
run_test "Elasticsearch (v1.7.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 1 https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.tar.gz"
run_test "Elasticsearch (v2.3.5) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 2 https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.3.5/elasticsearch-2.3.5.tar.gz"
run_test "Elasticsearch (v5.1.2) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 5 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.2.tar.gz"
-run_test "Elasticsearch (v6.3.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 6 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz"
+# Disabled because of test instability, see FLINK-10361
+#run_test "Elasticsearch (v6.3.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 6 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz"
run_test "Quickstarts Java nightly end-to-end test" "$END_TO_END_DIR/test-scripts/test_quickstarts.sh java"
run_test "Quickstarts Scala nightly end-to-end test" "$END_TO_END_DIR/test-scripts/test_quickstarts.sh scala"
@@ -70,9 +71,5 @@ run_test "State TTL RocksDb backend end-to-end test" "$END_TO_END_DIR/test-scrip
run_test "SQL Client end-to-end test" "$END_TO_END_DIR/test-scripts/test_sql_client.sh"
-run_test "Heavy deployment end-to-end test" "$END_TO_END_DIR/test-scripts/test_heavy_deployment.sh"
-
-run_test "ConnectedComponents iterations with high parallelism end-to-end test" "$END_TO_END_DIR/test-scripts/test_high_parallelism_iterations.sh 25"
-
printf "\n[PASS] All tests passed\n"
exit 0
diff --git a/splits/split_misc_hadoopfree.sh b/splits/split_misc_hadoopfree.sh
index 3d66896..8281300 100755
--- a/splits/split_misc_hadoopfree.sh
+++ b/splits/split_misc_hadoopfree.sh
@@ -52,7 +52,8 @@ run_test "Stateful stream job upgrade end-to-end test" "$END_TO_END_DIR/test-scr
run_test "Elasticsearch (v1.7.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 1 https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.tar.gz"
run_test "Elasticsearch (v2.3.5) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 2 https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.3.5/elasticsearch-2.3.5.tar.gz"
run_test "Elasticsearch (v5.1.2) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 5 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.2.tar.gz"
-run_test "Elasticsearch (v6.3.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 6 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz"
+# Disabled because of test instability, see FLINK-10361
+#run_test "Elasticsearch (v6.3.1) sink end-to-end test" "$END_TO_END_DIR/test-scripts/test_streaming_elasticsearch.sh 6 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz"
run_test "Quickstarts Java nightly end-to-end test" "$END_TO_END_DIR/test-scripts/test_quickstarts.sh java"
run_test "Quickstarts Scala nightly end-to-end test" "$END_TO_END_DIR/test-scripts/test_quickstarts.sh scala"
[flink] 08/11: [hotfix] Enable parallel building
Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 01340fe6790fb5c773fe16cec051dc9d6af95f81
Author: Timo Walther <tw...@apache.org>
AuthorDate: Thu Nov 8 09:31:47 2018 +0100
[hotfix] Enable parallel building
---
nightly.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/nightly.sh b/nightly.sh
index 19bc7b5..c2d9ff6 100755
--- a/nightly.sh
+++ b/nightly.sh
@@ -44,7 +44,7 @@ LOG4J_PROPERTIES=${FLINK_DIR}/tools/log4j-travis.properties
MVN_LOGGING_OPTIONS="-Dlog.dir=${ARTIFACTS_DIR} -Dlog4j.configuration=file://$LOG4J_PROPERTIES -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn"
MVN_COMMON_OPTIONS="-nsu -B -Dflink.forkCount=2 -Dflink.forkCountTestPackage=2 -Dfast"
-MVN_COMPILE_OPTIONS="-DskipTests"
+MVN_COMPILE_OPTIONS="-T1C -DskipTests"
git clone --single-branch -b ${BRANCH} https://github.com/${REMOTE}/flink
[flink] 06/11: [hotfix] Remove duplicate test execution
Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 31e8d61100e7ec73f8b10017bba3c68d6e81fd9f
Author: Timo Walther <tw...@apache.org>
AuthorDate: Wed Nov 7 10:27:15 2018 +0100
[hotfix] Remove duplicate test execution
---
splits/split_misc.sh | 2 --
1 file changed, 2 deletions(-)
diff --git a/splits/split_misc.sh b/splits/split_misc.sh
index a9ef821..abc1bcb 100755
--- a/splits/split_misc.sh
+++ b/splits/split_misc.sh
@@ -68,8 +68,6 @@ run_test "Avro Confluent Schema Registry nightly end-to-end test" "$END_TO_END_D
run_test "State TTL Heap backend end-to-end test" "$END_TO_END_DIR/test-scripts/test_stream_state_ttl.sh file"
run_test "State TTL RocksDb backend end-to-end test" "$END_TO_END_DIR/test-scripts/test_stream_state_ttl.sh rocks"
-run_test "Run kubernetes test" "$END_TO_END_DIR/test-scripts/test_kubernetes_embedded_job.sh"
-
run_test "SQL Client end-to-end test" "$END_TO_END_DIR/test-scripts/test_sql_client.sh"
run_test "Heavy deployment end-to-end test" "$END_TO_END_DIR/test-scripts/test_heavy_deployment.sh"
[flink] 01/11: Integrate scala 2.12 e2e tests
Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 4756ccf5789599d64f128e7c2766ffbab4542bf9
Author: zentol <ch...@apache.org>
AuthorDate: Wed Nov 7 09:12:17 2018 +0100
Integrate scala 2.12 e2e tests
---
.travis.yml | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index 70e822c..13e504a 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -75,6 +75,36 @@ matrix:
- BRANCH="release-1.7"
- PROFILE="-Dhadoop.version=2.8.3"
- SCRIPT="split_kubernetes_e2e.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12 -De2e-metrics"
+ - SCRIPT="split_misc.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
+ - SCRIPT="split_ha.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
+ - SCRIPT="split_sticky.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
+ - SCRIPT="split_checkpoints.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
+ - SCRIPT="split_docker_e2e.sh"
+ - env:
+ - REMOTE="apache"
+ - BRANCH="master"
+ - PROFILE="-Dhadoop.version=2.8.3 -Dscala-2.12"
+ - SCRIPT="split_kubernetes_e2e.sh"
git:
depth: 100
[flink] 03/11: add executable permission
Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch cron-1.7-e2e
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 8e8aa879ed34b443f92e2b385163c95e6eb85d94
Author: zentol <ch...@apache.org>
AuthorDate: Sun Nov 11 17:51:52 2018 +0100
add executable permission
---
splits/split_misc_hadoopfree.sh | 0
1 file changed, 0 insertions(+), 0 deletions(-)
diff --git a/splits/split_misc_hadoopfree.sh b/splits/split_misc_hadoopfree.sh
old mode 100644
new mode 100755